Full-text (PDF) ResearchGate, the professional network for scientists.

The time required to conduct drug and alcohol screening has been a major barrier to its implementation in mainstream healthcare settings. Because patient self-administered tools are potentially more efficient, we translated the Alcohol, Smoking and Substance Involvement Screening Test (ASSIST) into an audio guided computer assisted self interview (ACASI) format. This study reports on the test-retest reliability of the ACASI ASSIST in an adult primary care population. Adult primary care patients completed the ACASI ASSIST, in English or Spanish, twice within a 1–4 week period.

Among the 101 participants, there were no significant differences between test administrations in detecting moderate to high risk use for tobacco, alcohol, or any other drug class. Substance risk scores from the two administrations had excellent concordance (90–98%) and high correlation (ICC 0.90–0.97) for tobacco, alcohol, and drugs. The ACASI ASSIST has good test-retest reliability, and warrants additional study to evaluate its validity for detecting unhealthy substance use.

INTRODUCTION Substance abuse leads to more death and disability than any other preventable health condition, yet only a minority (~11%) of those with drug and alcohol use disorders receive specialized treatment (;;; ). Primary healthcare providers often constitute the only health system contacts for this population, particularly for those who have not yet developed severe drug use disorders (;; ). There is therefore intense interest in identifying and addressing substance use in general healthcare settings, but significant challenges exist to integrating this practice into regular medical care.

Screening and brief intervention (SBI) describes an approach that seeks to identify and provide interventions for problematic substance use in mainstream healthcare settings (;;, ). It involves streamlined screening and assessment, followed by brief interventions that can be carried out by medical providers in the course of a regular office visit. A major impediment to wide-scale implementation of SBI for drugs other than alcohol and tobacco is lack of a paradigm for routine and efficient screening and assessment for drug use in general medical settings, (; )and it often goes undetected as a result (;; ). A number of screening and assessment approaches have been developed to identify unhealthy substance use.

Following the tobacco Clinical Practice Guideline, many practices systematically screen for tobacco use at each visit, using strategies such as inclusion of tobacco use in the vital signs, or documentation of tobacco use status in the electronic health record (). The AUDIT and AUDIT-C are widely recommended for alcohol screening and assessment, and have been implemented as part of routine primary care in large health systems including the Veterans Administration (; ). Screening and assessment tools for other drugs, including a single-item drug screen and a 10-item version of the DAST, have been developed and validated in primary care populations (; ), but have yet to be widely adopted in practice. In busy medical practice settings, identification of drug and alcohol use could be facilitated by having a unified screening paradigm that integrates tobacco, alcohol, and drugs in a brief screen that generates clinically relevant results. One such instrument is the “Alcohol, Smoking and Substance Involvement Screening Test (ASSIST),” a validated structured interview that was developed by the World Health Organization for use in general healthcare settings (; ). However, the ASSIST has proven difficult to incorporate into routine care (; ), in part because it takes approximately 5–15 minutes of face-to-face interaction with the patient to administer, has complex skip patterns, and bases its calculation of risk on a scoring system that must be computed by the interviewer. Yet the ASSIST has some clear advantages for use in SBI programs, in comparison to briefer assessment instruments.

Specifically, it includes tobacco and alcohol alongside other drugs; gives a substance specific risk score (low, moderate, high) that can be used to guide the clinical intervention; and screens for injection drug use (;;; ). The ASSIST identifies lifetime as well as current use, which may be clinically relevant for preventive care such as identification of individuals at risk for developing health sequelae of prior use (e.g. Hepatitis C, chronic obstructive pulmonary disease (COPD)), or for identifying individuals who are at risk of relapse and may need closer monitoring than those with no history of substance use.

These features of the ASSIST have the potential to streamline the substance use history and assessment of the primary care provider, and maximize use of the provider’s time for delivering clinical interventions. To increase the potential feasibility of using the ASSIST in medical settings, we adapted the previously validated interviewer-administered ASSIST to a patient self-administered format using audio guided computer assisted self interview (ACASI) technology. ACASI instruments make it possible to support even patients with limited reading ability in completing self-administered questionnaires because all relevant text is read aloud in real time, and response options are clearly indicated using symbols. An ACASI version of the ASSIST could be completed in the waiting room, with results conveyed to the provider at the start of the visit. Such an innovation could reduce barriers to implementing routine screening and assessment for substance use in the primary care setting, by providing an instrument that more efficiently fits into clinical workflows (). A self-administered screening approach also has the potential to reduce the stigma associated with disclosure of substance use in a face-to-face interview, and could potentially make patients feel more comfortable and respond more honestly (;; ). Unlike interviewer-administered screening tools which rely on individuals delivering the items exactly as written (), the self-administered approach delivers validated screening instruments consistently and with high fidelity, even in the context of routine clinical care.

Computerized self interview approaches, particularly ACASI, have proven sensitive for detecting stigmatized behaviors, have comparable validity to traditional interview formats, are easily adapted to multiple languages, and can be integrated into medical settings (;;;;;; ). Notably, an ACASI version of the AUDIT was shown to be feasible, acceptable to patients, and equally good as an interviewer- or ‘pencil and paper’-administered version at detecting problem drinking among English speaking patients (;; ). This suggests that an ACASI ASSIST could be a viable option in general healthcare settings, but with the exception of one small study in an English-speaking college student population, the validity of this approach has not been tested (). As a preliminary step in a line of research investigating the feasibility and validity of an ACASI ASSIST for substance use screening and assessment in primary care, we examined its test-retest reliability. Assessment of test-retest reliability is an important early step in instrument development, but has sometimes been overlooked in the development of substance use screening tools (;; ). Test-retest reliability measures the consistency of an instrument’s measurements, and is usually undertaken prior to studies of diagnostic accuracy (sensitivity, specificity).

Test-retest reliability was studied by the WHO ASSIST Working Group for the interviewer-administered ASSIST, as part of the process of developing that instrument (). In that international study, a total of 236 participants recruited from primary care and substance abuse treatment settings completed the ASSIST twice in a 1–3 day period. The ASSIST items demonstrated good test-retest reliability, with average kappa coefficients across the ASSIST items for each substance class ranging from 0.61 to 0.78.

This earlier work informs the present study, which examines test-retest reliability of the ACASI ASSIST in a U.S. Adult primary care population. Participants and recruitment The study was conducted in the adult primary care medicine clinic of a large municipal hospital in New York City.

The first enrollment period was April-August 2011, during which time 85 participants were enrolled. Marilyn Manson Lest We Forget Rapidshare Downloads here. Because the rate of follow-up was lower than anticipated, to increase the sample size for this test-retest reliability analysis we added a second enrollment period, January-April 2012, during which 61 participants were enrolled. While the parent study was broader in scope and included additional study procedures, the test-retest reliability of the ACASI ASSIST was the primary focus, and is the only aspect of the study presented here. All recruitment was from the clinic waiting area. Individuals were approached consecutively, screened for eligibility, and offered participation in the study regardless of their demeanor.

There was no advertisement, and all recruitment was done by three research assistants (RAs) during the initial enrollment period, and by one RA during the second enrollment period. Eligible individuals were required to be current clinic patients, age 18–65.

In the initial enrollment period we oversampled patients anticipated to have greater difficulty using the ACASI instrument by recruiting a pre-set number of participants who met one of the following additional criteria: less than high school education, 50–65 years old, or primary language Spanish. In the second enrollment period, only individuals fluent in English were eligible, and a purposeful sampling approach was used to achieve approximately equal numbers of male and female participants. Potential participants received a written information sheet, and verbal consent for participation was obtained prior to any study assessments.

All participants received a modest cash incentive plus a round-trip transit card for participating, and were offered referral information for tobacco and alcohol/drug treatment services. The institutional review board of [name blinded] reviewed and approved of all study procedures. Study Instrument: ACASI ASSIST The Alcohol, Smoking and Substance Involvement Screening Test (ASSIST) was developed by the World Health Organization (WHO) and a global consortium of substance abuse researchers as a screening tool for substance use and addiction, applicable to primary care patients in diverse populations ().

It has been used for both clinical screening and for research purposes, and has been translated and validated by WHO in multiple languages. The ASSIST provides a substance specific risk stratification that can guide clinical interventions; typically monitoring and prevention for low risk, brief intervention for moderate risk, and treatment or treatment referral for high risk use. The ASSIST was developed and adapted over a five-year period by WHO, leading up to the ASSIST V3.0, which was the basis for the ACASI ASSIST used in our study. ASSIST V3.0 is a brief structured interview covering nine substances (tobacco, alcohol, cannabis, cocaine, stimulants, inhalants, sedatives, hallucinogens, opioids), that assesses lifetime use, current use, consequences of use, and failure to stop or cut down. The first item in the ASSIST asks about lifetime use for each substance. Individuals who screen negative for lifetime use of all substances receive no further questions.

Those who screen positive for lifetime use go on to complete the subsequent items pertaining to the substances for which they reported lifetime use. For example, an individual who reported lifetime use of marijuana only would receive the follow-up questions only with reference to marijuana. Item 2 of the ASSIST assesses current (past 3 months) use, while items 4–7 assess various problems related to substance use, and item 8 asks about any lifetime or current injection drug use. Individuals who report lifetime use but no current use of a substance are still administered items 6–7 for that substance, as well as item 8 on injection drug use.

Item 6 asks about whether close contacts have ever expressed concern about the individual’s use, while item 7 asks if they have ever tried and failed to cut down on their use of that substance. The ASSIST instrument used in our study maintained all of the structural features of the WHO ASSIST V3.0, but was adapted to include two additional substances: prescription opioids and prescription stimulants. This change is consistent with the inclusion of these substance classes in the NIDA-modified ASSIST ().

The survey preamble, questions, and format were otherwise unchanged from the ASSIST V3.0. We translated the modified ASSIST to audio computer assisted self interview (ACASI) format, in English and in Spanish, using QDS Software (Nova Research Co.). This ‘ACASI ASSIST’ was administered on touch-screen tablet computers, with headphones. Participants were guided by voice instruction and written text on the computer screen in the same way that the ASSIST would be administered by an interviewer who was reading the instrument verbatim, maintaining identical language and skip patterns to those specified in the WHO ASSIST V3.0. The preamble and ASSIST items were delivered in their entirety, and written text on the computer screen was identical to the words of the voice instruction. Study Procedures Assessments were conducted in either English or Spanish, depending on the preferred language of the participant. At the first visit (Time 1), participants met with the RA in a private room to complete the ACASI ASSIST.

For participants recruited during the first enrollment period, the ACASI ASSIST was followed by the interviewer administered (IA) ASSIST at Time 1. Participants recruited during the second enrollment period did not receive the IA ASSIST, but did receive a 4-item self- substance use screening questionnaire prior to the ACASI ASSIST. The 4-item screener asked about past 12 months use of tobacco, alcohol, illicit drugs, and prescription drugs, and was administered in this study to provide an assessment of this instrument’s test-retest reliability (only results from the ACASI ASSIST are presented here). Demographic information was collected at the end of the first visit, and all participants were scheduled to return within 1–3 weeks for a second visit (Time 2).

Those who returned for the Time 2 visit slightly outside of the designated window (1 individual at 6 days, 10 individuals at 21–27 days) were still included. At the second visit, the ACASI ASSIST was administered, and the IA ASSIST was not administered. The RA waited outside the room while participants completed the ACASI ASSIST, to assure privacy and encourage independent completion of the questionnaire. Participants who needed assistance had the option of clicking a ‘help’ button that provided additional explanation of the item they were answering, and they were able to call the RA into the room to request technical assistance. During the first enrollment period, requests for assistance using the ACASI were tracked at Time 1 by the RA, who filled out a form indicating whether the participant requested assistance, and of what type (i.e.

Help using the computer, comprehending the questions, or other assistance). Time for each ACASI ASSIST interview was recorded automatically by the computer. Risk scores The ASSIST global scores and substance specific involvement scores (SSIS) were calculated using standard ASSIST methodology (). These scores included the two substance categories (prescription opioids and stimulants) that we added to the original ASSIST V3.0. The global score represents the sum of all responses to ASSIST Question 1–8, and has a potential range of 0 to 498. Following the standard approach to scoring the ASSIST, the SSIS is the sum of responses to ASSIST Questions 2–7, for each substance, and has a potential range of 0 to 39. Because the ASSIST instrument specifies that individuals who report no current use of a substance should still receive follow-up questions about whether a ‘friend or relative or anyone else ever expressed concern’ about that their use (Question 6), or if they ‘ever tried and failed’ to reduce their use (Question 7), the SSIS may be greater than zero even for individuals who are not current users of that substance.

ASSIST scores were further aggregated into a summary ‘drug’ category that includes all substances other than tobacco and alcohol (and includes misuse of prescription medications). The summary ‘drug score’ was calculated from the sum of individual SSIS across all drugs, and has a potential range of 0 to 390. Risk level WHO-recommended cutoffs were used to determine level of risk (low, moderate, or high) from the SSIS for each substance. For alcohol, a score of 0–10 constitutes low risk, 11–26 moderate risk, and 27+ high risk.

For tobacco and all other drugs, a score of 0–3 constitutes low risk, 4–26 moderate risk, and 27+ high risk. Low risk individuals are not considered to require intervention for their substance use, whereas for those with moderate or high risk an intervention is indicated (). In our analyses, we collapsed the moderate and high risk levels into a single ‘moderate-high risk’ category. The choice was made to combine moderate and high risk users because high risk substance use was relatively infrequent in this study population, and we felt it was most relevant to focus on distinguishing between individuals whose substance use requires clinical intervention (due to moderate or high risk use) versus those who do not require intervention (due to low risk use). Additional tables showing the concordance of risk scores for moderate risk and high risk use alone are included in the and ). Level of risk for the aggregate ‘drugs’ category was based on the highest risk level for any substance included in that category.

For example, an individual with low risk marijuana use and moderate risk cocaine use would be classified as ‘moderate-high risk’ in the drugs category. Statistical Analysis The analysis was restricted to individuals who completed both study visits.

One participant who skipped all ASSIST items at both visits was excluded from the analysis. To characterize the study population, we used descriptive statistics (means and proportions) to summarize participants’ demographic characteristics, and to describe the prevalence of substance use based on responses given on the ACASI ASSIST at Time 1 or Time 2. Comparing participants recruited during the first versus the second enrollment period, we did not detect any statistically significant (P.

Analysis of agreement for classification of risk level To evaluate test-retest reliability, we began by examining the level of agreement in classifying individuals as low versus moderate-high risk users, for all 12 substance classes queried by the ACASI ASSIST. We first estimated the prevalence of low risk versus moderate-high risk use for each substance class at Time 1 and Time 2, then calculated the proportion of individuals who had either concordant risk levels (low risk at both time points or moderate-high risk at both time points), an increase in risk level (from low risk at Time 1 to moderate-high risk at Time 2), or a decrease in risk level (from moderate-high risk at Time 1 to low risk at Time 2). To test the degree of concordance between Time 1 and Time 2 results, we examined the proportion of concordant results for individuals having low risk versus moderate-high risk use using McNemar tests, with P-values.05 considered indicative of no significant change in score between Time 1 and Time 2.

We additionally examined the correlation of results at Time 1 and Time 2, for low versus moderate-high risk use, using Cohen’s Kappa. Kappa coefficients were computed for all substance classes having prevalence greater than 20% in the study population. They were not calculated for lower prevalence substance classes because the dependence of Kappa on prevalence can compromise its interpretation in conditions of markedly high or low prevalence (; ). The Kappa coefficients were interpreted using the standard guidelines of 0.20 or less indicating poor agreement, 0.21 to 0.40 fair agreement, 0.41 to 0.60 moderate agreement, 0.61 to 0.80 substantial agreement, and 0.81 to 1.0 almost perfect or perfect agreement (). While only the analysis of agreement for the combined category of moderate-high risk substance use is shown in the manuscript, the same analytic steps were applied to two separate analyses of moderate risk and of high risk use only. Results of those analyses are included as an and ).

The same analysis was applied to the three pre-specified subgroups of individuals who were anticipated, based on prior studies, to have greater difficulty using the ACASI instrument (;; ). These subgroups were: those having less than high school education; those 50 years old or greater; and those whose primary language was Spanish. Analysis of agreement for ASSIST scores In a second approach to our analysis of agreement, we examined correlation between the ACASI ASSIST scores generated at Time 1 and Time 2, using Intraclass Correlation Coefficients (ICCs). We calculated ICCs, assessing agreement for repeated ACASI ASSIST measurements, for the following scores, all of which are continuous variables: a) global score, b) SSIS for each substance, and c) a summary ‘drug score,’ which included all substances other than tobacco and alcohol. We additionally computed the ICC for the global score limited to the items that comprise the WHO ASSIST V3.0, and thus did not include the prescription stimulant and prescription opioid items. The ICC calculations used a single measurement, absolute agreement definition, two-way mixed model. ICCs were interpreted using recommended guidelines of Cicchetti for reliability of clinical instruments: less than 0.40 indicates poor agreement, 0.40 to 0.59 fair agreement, 0.60–0.74 good agreement, and 0.75 to 1.00 excellent agreement ().

We considered using the approach of Bland and Altman as an additional evaluation of agreement, but were unable to apply it here because its interpretation requires that distribution of differences between the measures be normally distributed, which was not the case for all of our difference scores (). All analyses were conducted in IBM SPSS Statistics 20. Characteristics of the study population Of the 146 individuals who completed assessments at Time 1, 101 (69%) returned and completed assessments for Time 2.

Individuals from the initial enrollment period had a lower rate of follow up visits than those from the second enrollment period (55% vs. Those who failed to return at Time 2 did not differ significantly in prevalence of current drug or alcohol use, age, sex, education, or income, but were more likely to be foreign born, Hispanic, and to speak Spanish as their primary language. The mean time that elapsed between Time 1 and Time 2 assessments was 12 days (SD=6, range 6–27 days), and 72% of participants returned within 14 days. The average time required to complete the ACASI ASSIST on the first administration was 5.4 minutes (range 1.5 – 17.7 minutes). Data on requests for assistance in completing the ACASI ASSIST was captured for the first 35 participants. Among this group, 5 (14%) requested some type of assistance (3 asked for help using the computer, 2 for help with comprehension of the questions).

After removing from the analysis the single individual who skipped all items of the ASSIST, responses were 100% complete for all participants. Characteristics of the 101 subjects who completed two visits and were included in this analysis are presented in. Thirty-six percent of participants were born outside the United States, and 26% had a primary language other than English. Thirty percent of participants did not complete high school, while 22% had a high school diploma or GED.

3.2 Self-reported prevalence of substance use and risk level Prevalence of lifetime and past 3 months substance use, based on responses to questions 1 and 2 of the ACASI ASSIST, is shown in. As indicated in the table, ‘drugs’ refers to all substances other than alcohol and tobacco. Fifty-nine percent of participants reported lifetime use of tobacco, while 83% reported lifetime use of alcohol, and 74% reported lifetime use of at least one other drug.

Rates of current use were 32% for tobacco, 67% for alcohol, and 42% for drugs. In the drug category, marijuana was the substance with highest prevalence of lifetime use, while cocaine was the substance with highest prevalence of current use. The majority (59%) of current drug users also reported current use of tobacco, and 38% reported current use of alcohol at least weekly.

For the 33 individuals who reported no current alcohol use, 27% were current users of at least one drug (data not shown). Prevalence and risk level by substance at Time 1, based on ACASI ASSIST responses (N=101) Risk level was specified using the standard ASSIST cutoffs designating low risk (not requiring clinical intervention) and moderate-high risk (requiring clinical intervention).

For each substance queried, between 65% and 96% of participants had ASSIST scores indicating low risk use, while 4–28% had moderate-high risk use (). Some individuals with ASSIST scores indicating moderate-high risk use did not have current (past 3 months) use. For example, although 24 individuals had moderate-high risk use of marijuana, only 18 reported current marijuana use. Concordance of risk level classification at Time 1 and Time 2 The first measure of test-retest reliability was concordance in classification of risk level between Time 1 and Time 2. Overall, 93% of individuals had a concordant risk level at Time 1 and Time 2, meaning that they either scored ‘low risk’ for all substances at both administrations of the ACASI ASSIST, or scored ‘moderate-high risk’ for at least one substance at both administrations. Two individuals (2%) had an increased risk at Time 2, while 5 (5%) had a decreased risk at Time 2. Shows change in risk category by substance.

For example, in the summary drug category, 53 individuals had a low risk score for each drug on the ACASI ASSIST at Time 1, and 56 individuals had a low risk score for each drug at Time 2, while 48 individuals had a moderate-high risk score for at least one drug at Time 1, and 45 at Time 2. There were no statistically significant changes (P. Correlation of ASSIST scores at Time 1 and Time 2 We examined correlation of ASSIST scores at Time 1 and Time 2 for the ASSIST global score (sum across all substances and all items), and for tobacco, alcohol and drug scores. As shown in, we observed excellent agreement between Time 1 and Time 2 scores for each of these substance categories using the ICC: global score ICC=0.968; tobacco score ICC=0.900; alcohol score ICC=0.904; and drug score ICC=0.969.

The ICC for global score was similar with and without inclusion of the prescription stimulant and prescription opioid items (ICC=0.965 without these items, and ICC=0.968 with these items included). All correlations were significant at the P. DISCUSSION The ACASI ASSIST demonstrated overall good test-retest reliability and required on average only five minutes for primary care patients to complete. These findings are particularly important to consider in the context of our study population, which included a high proportion of individuals who might be anticipated to have difficulty with a self-administered computerized screening tool. Many of our participants were born outside the US, had Spanish as their primary language, or had low levels of formal education. Based on prior studies, these populations may be anticipated to have more difficulty with a computer self-administered instrument (;; ). We nonetheless found overall good test-retest reliability among participants, including among our subgroups of Spanish speakers, those with less than high school education, and individuals over 50 years old.

We believe that the use of ACASI technology, together with a touchscreen computer, was important to making this self-administered instrument feasible in our study population. We examined two types of test-retest reliability; concordance, and correlation.

Prior to our study, only correlation (using Cohen’s Kappa or Pearson’s r) had been used to assess test-retest reliability of the ASSIST (; ). Because concordance assesses change in the classification of risk, which is what drives the subsequent clinical intervention, it is arguably the more relevant measure for this type of screening test (). Bennetts Cosmetics Formulary Pdf Free here. In the clinical setting it is more important to know, for example, whether the instrument is reliable in distinguishing between low risk use of cocaine (requiring little or no clinical intervention) and moderate-high risk use of cocaine (requiring brief intervention and potential treatment referral), than it is to know whether the substance specific risk score changed from a score of 5 to a score of 7. We observed good to excellent correlation between ASSIST scores at the two administrations using the kappa statistic for low versus moderate-high risk use and the ICC for ASSIST scores, but were limited by being able to examine correlation only in the substance classes that had sufficient prevalence to reliably compute these statistics.

Unfortunately, our measures of correlation for the test-retest reliability of the ACASI ASSIST cannot be directly compared to those derived for the interviewer-administered WHO ASSIST in an earlier test-retest reliability study, because that study used an earlier version of the ASSIST instrument (). The prevalence of substance use in our primary care population is higher than rates in the general population (). While rates of moderate-high risk use were much lower than rates of any use or low risk use in all substance classes, we still found that nearly half of our population had moderate-high risk use of drugs, indicating that they should receive a clinical intervention. This prevalence of moderate-high risk use was higher than that estimated in an earlier prevalence study using an interviewer-administered ASSIST in the same primary care clinic, perhaps due to differences in the recruitment strategy, sample, and mode of administration of the ASSIST (). Nonetheless, high prevalence in our study was comparable to that found in other safety net adult primary care settings (;; ).

In the validation study of a single item screening question for drugs (), for example, 35% of the primary care study population screened positive for past year drug use, while in our study 42% reported current drug use. We also found that rates of tobacco use were much higher among participants in our study than for the general NYC population (32% versus 14%) ().

Notably, a screening approach that focused on alcohol use alone would have failed to identify a significant proportion of risk in our population. Among low-risk alcohol users, one-third had moderate-high risk drug use, which is likely to have gone undetected with an alcohol screening tool. These findings indicate that substance use screening in a safety net primary care population may identify a significant burden of clinically relevant use, and that using a screening tool that integrates screening for alcohol and drugs may be necessary to identify the patient population having risky substance use behaviors. Limitations of the study Our study has several limitations. Despite the overall high prevalence of substance use, given the small sample size we had limited ability to analyze results for some specific drug classes, including hallucinogens, prescription stimulants, methamphetamine, and inhalants, use of each of which was reported by fewer than 10 individuals.

Our sample also included relatively few individuals with high-risk substance use, which limited our ability to look at the ACASI ASSIST’s test-retest reliability among those with the most severe substance use. Low prevalence is likely to be a limitation for any study conducted in a general primary care population. However, the drawbacks to testing an instrument in a relatively low prevalence population should be weighed against the benefits of testing an instrument in the population for which it is intended to be used – in this case, a general adult primary care population. Generalizability of our findings is limited by having conducted the study at a single adult primary care clinic site. Our sample reflects the characteristics of a diverse urban safety net clinic population.

While this may impact the generalizability of our findings to other clinical settings, it also represents the type of clinical population that may be a higher priority for substance use screening, due to relatively high prevalence of drug and alcohol use. Testing the ACASI ASSIST in this context could thus be interpreted as a strength of our approach. Self-administered tools such as the ACASI ASSIST are likely to be less acceptable to individuals who have greater difficulty completing them, and this could have influenced our results through loss to follow-up. Those who failed to return at Time 2 may have been less comfortable using the ACASI ASSIST because they were Spanish speaking, although it was delivered in Spanish. A prior study of computer self-administered substance use screening for alcohol in primary care indicated poorer performance in Spanish versus English speakers, even when administered in the participant’s first language (), although more recent clinical experience with this modality has demonstrated good feasibility (; ). Though we did not detect significant differences in results for our Spanish speaking subgroup, further testing in this population may be indicated. Individuals with less than high school education levels also had lower rates of follow-up at Time 2 compared to the study population as a whole, though the difference in follow-up rates did not reach the level of statistical significance.

It is possible that those with lower education level were less comfortable with the ACASI ASSIST, and for that reason failed to return. In the future, qualitative interviews could contribute to our understanding of how acceptable the ACASI ASSIST is for individuals with lower education or literacy levels. In a test-retest study of an instrument that measures an impermanent characteristic such as substance use, it is possible that differences between Time 1 and Time 2 results reflect actual changes in use, which we were not able to evaluate in the present study. We chose a brief timeframe in order to reduce the likelihood that individuals would have significant changes in their actual substance use. Some have postulated that research assessments alone can produce changes in self-reported substance use (; ), and it is possible that reactivity to the research assessments at Time 1 could have influenced substance use behavior and reporting at Time 2. We tried to minimize this source of bias by limiting the assessments given at Time 1, but individuals enrolled during the first enrollment period did complete the interviewer-administered ASSIST (following their completion of the ACASI ASSIST) at Time 1.

Yet even among these individuals, the ACASI ASSIST demonstrated good test-retest reliability. Given that the instrument relies on self-reported responses, there is also a risk that responses could be biased by individuals who remembered how they answered to items on the first administration, and tried to give the same answer on the second administration. Conclusions Our study indicates that a self-administered ACASI ASSIST has good test-retest reliability in a primary care population. The ACASI ASSIST has the potential to be a useful tool for conducting screening and a clinically-oriented structured assessment for unhealthy substance use in primary care.

However, before recommending broad implementation of the ACASI ASSIST, its validity, especially as regards its sensitivity and specificity in detecting unhealthy substance use, must first be evaluated. This will require additional studies that are able to compare the ACASI ASSIST to reference standard measures. Generally speaking, computer self-administered tools have promise for facilitating substance use screening, assessment, and even treatment interventions in time-pressured medical settings (;;;; ). Brief screeners such as the single-item drug screening question (), may be adequate for identifying individuals with unhealthy drug use, but require further assessment to determine what drugs are being used, and at what level of risk, in order to guide the clinical intervention. The ASSIST has the advantage of providing this information, as well as integrating the screening and assessment process for tobacco, alcohol, and other drugs.

Yet the interviewer-administered ASSIST is not feasible in most primary care environments because it requires an interviewer who can devote up to 15 minutes to its administration. A computer self-administered assessment could be completed on a tablet computer or kiosk in the waiting area, or even at home via an internet portal, prior to the medical visit, and have its results incorporated into the electronic health record at the point of care. A potential disadvantage of using a self-administered tool is that some patients may feel more comfortable responding to questions posed by an interviewer, and self-administration eliminates the possibility of developing of rapport with the interviewer during the screening process. However, this may be counterbalanced by the possibility that a self-administered instrument can encourage reporting of stigmatized behavior (;; ) and has high fidelity, since it is not influenced by factors such as tone of voice, paraphrasing, or editorializing on the part of the person administering it. While self-administered questionnaires run the risk of being difficult for individuals with limited reading ability, ACASI technology has the potential to make these assessments feasible in diverse populations.

This approach warrants further study in the context of implementing substance use screening in primary care settings. Contributor Information Jennifer McNeely, Assistant Professor, Department of Population Health and Department of Medicine, Division of General Internal Medicine, New York University School of Medicine, 550 1 st Avenue, VZ30, 6 thFloor New York, NY 10016, Tel (212) 263-4975, Fax (646) 501-2706. Strauss, New York University College of Nursing. Shana Wright, Department of Population Health, New York University School of Medicine. John Rotrosen, Department of Psychiatry, New York University School of Medicine. Rubina Khan, Department of Population Health, New York University School of Medicine.

Lee, Department of Population Health, New York University School of Medicine. Gourevitch, Department of Population Health, New York University School of Medicine. • Ali R, Awwad E, Babor TF, Bradley F, Butau T, Farrell M Grp Who Assist Working. The alcohol, smoking and substance involvement screening test (ASSIST): development, reliability and feasibility. 2002; 97(9):1183–1194. [] • Babor Thomas F, McRee Bonnie G, Kassebaum Patricia A, Grimaldi Paul L, Ahmed Kazi, Bray Jeremy.

Screening, Brief Intervention, and Referral to Treatment (SBIRT): toward a public health approach to the management of substance abuse. 2007; 28(3):7–30. [] • Bernstein JA, Bernstein E, Heeren TC. Mechanisms of change in control group drinking in clinical trials of brief alcohol intervention: implications for bias toward the null. Drug Alcohol Rev. 2010; 29(5):498–507. Doi: 10.1111/j.1465-3362.2010.00174.x.

[] [] • Bertollo David N, Alexander Mary Jane, Shinn Marybeth, Aybar Jalila B. Innovations: clinical computing: an audio computer-assisted self-interviewing system for research and screening in public mental health settings. Psychiatr Serv. 2007; 58(6):743–745. [] • Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement.

1986; 1(8476):307–310. [] • Bland JM, Altman DG. Measuring agreement in method comparison studies. Stat Methods Med Res.

1999; 8(2):135–160. [] • Bradley KA, Lapham GT, Hawkins EJ, Achtmeyer CE, Williams EC, Thomas RM, Kivlahan DR. Quality concerns with routine alcohol screening in VA clinical settings.

J Gen Intern Med. 2011; 26(3):299–306. Doi: 10.1007/s11606-010-1509-4. [] [] [] • Bradley K, Berger D.

Screening for Unhealthy Alcohol Use. In: Saitz R, editor. Addressing Unhealthy Alcohol Use in Primary Care. New York: Springer; 2013. • Butler SF, Budman SH, Goldman RJ, Newman FL, Beckley KE, Trottier D, Cacciola JS.

Initial validation of a computer-administered Addiction Severity Index: the ASI-MV. Psychol Addict Behav. 2001; 15(1):4–12.

[] • Butler SF, Chiauzzi E, Bromberg JI, Budman SH, Buono DP. Computer-assisted screening and intervention for alcohol problems in primary care. Journal of Technology in Human Services. 2003; 21(3):1–19. • Center for Integrated Health Solutions. SBIRT: Screening, brief intervention and referral to treatment, opportunities for implementation and points for consideration. 2012 Retrieved May 28, 2013, from.

• Chan-Pensley E. Alcohol-Use Disorders Identification Test: a comparison between paper and pencil and computerized versions. Alcohol Alcohol.

1999; 34(6):882–885. [] • Cicchetti DV. Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology.

Psychological Assessment. 1994; 6(4):284–290. • D’Amico Elizabeth J, Paddock Susan M, Burnam Audrey, Kung Fuan-Yue.

Identification of and guidance for problem drinking by general medical providers: results from a national survey. 2005; 43(3):229–236. [] • Fiore MC, Jaen CR, Baker TB, Bailey WC, Bennett G, Benowitz NL, Treati Clinical Practice Guideline. A Clinical Practice Guideline for Treating Tobacco Use and Dependence: 2008 Update - A US Public Health Service report. American Journal of Preventive Medicine.

2008; 35(2):158–176. Doi: 10.1016/j.amepre.2008.04.009. [] [] [] • Friedmann PD, McCullough D, Saitz R. Screening and intervention for illicit drug abuse: a national survey of primary care physicians and psychiatrists. Arch Intern Med. 2001; 161(2):248–251. [] • Harris SK, Csemy L, Sherritt L, Starostova O, Van Hook S, Johnson J, Knight JR.

Computer-facilitated substance use screening and brief advice for teens in primary care: an international trial. 2012; 129(6):1072–1082. Doi: 10.1542/peds.2011-1624. [] [] [] • Humeniuk R. Validation of the Alcohol, Smoking and Substance Involvement Screening Test (ASSIST) and pilot brief intervention: A technical report of phase II findings of the WHO ASSIST Project. 2008 Retrieved May 28, 2013, from. • Humeniuk R, Ali R, Babor TF, Farrell M, Formigoni ML, Jittiwutikarn J, Simon S.

Validation of the Alcohol, Smoking And Substance Involvement Screening Test (ASSIST) Addiction. 2008; 103(6):1039–1047.

Doi: 1010.1111/j.1360-0443.2007.02114.x. Epub 02008 Mar 02128. [] • Humeniuk R, Ali R, Babor T, Souza-Formigoni ML, de Lacerda RB, Ling W, Vendetti J. A randomized controlled trial of a brief intervention for illicit drugs linked to the Alcohol, Smoking and Substance Involvement Screening Test (ASSIST) in clients recruited from primary health-care settings in four countries.

2012; 107(5):957–966. Doi: 10.1111/j.1360-0443.2011.03740.x. [] [] • Kim J, Dubowitz H, Hudson-Martin E, Lane W. Comparison of 3 data collection methods for gathering sensitive and less sensitive information. Ambulatory Pediatrics. 2008; 8(4):255–260. Doi: 10.1016/j.ambp.2008.03.033.

[] [] • Landis JR, Koch GG. The measurement of observer agreement for categorical data. 1977; 33(1):159–174.

[] • Lee Joshua D, Delbanco B, Wu E, Gourevitch MN. Substance use prevalence and screening instrument comparisons in urban primary care. 2011; 32(3):128–134. Doi: 10.107.2011.562732. [] [] • Lotfipour S, Cisneros V, Chakravarthy B, Barrios C, Anderson CL, Fox JC, Vaca FE. Assessment of readiness to change and relationship to AUDIT score in a trauma population utilizing computerized alcohol screening and brief intervention. 2012; 33(4):378–386.

Doi: 10.107.2011.645951. [] [] [] • Madras BK, Compton WM, Avula D, Stegbauer T, Stein JB, Clark HW. Screening, brief interventions, referral to treatment (SBIRT) for illicit drug and alcohol use at multiple healthcare sites: comparison at intake and 6 months later. Drug Alcohol Depend. 2009; 99(1–3):280–295. Doi: 10.1016/j.drugalcdep.2008.08.003.

[] [] [] • McCance-Katz EF, Satterfield J. SBIRT: a key to integrate prevention and treatment of substance abuse in primary care. 2012; 21(2):176–177. Doi: 10.1111/j.1521-0391.2011.00213.x.

[] [] [] • McGinnis JM, Foege WH. Actual causes of death in the United States. 1993; 270(18):2207–2212. [] • McNeely J, Lee JD, Grossman E. Other Drug Use. In: Saitz R, editor.

Addressing Unhealthy Alcohol Use in Primary Care. New York: Springer; 2013. • Mdege ND, Lang J.

Screening instruments for detecting illicit drug use/abuse that could be useful in general hospital wards: a systematic review. Addict Behav. 2011; 36(12):1111–1119.

Doi: 10.1016/j.addbeh.2011.07.007. [] [] • Mokdad AH, Marks JS, Stroup DF, Gerberding JL.

Actual causes of death in the United States, 2000. 2004; 291(10):1238–1245. Doi: 10.1001/jama.2. [] [] • Murphy MK, Bijur PE, Rosenbloom D, Bernstein SL, Gallagher EJ.

Feasibility of a computer-assisted alcohol SBIRT program in an urban emergency department: patient and research staff perspectives. Addict Sci Clin Pract. 2013; 8(1):2. Doi: 10.1186/1940-0640-8-2.

[] [] [] • National Institute on Alcohol Abuse and Alcoholism (NIAAA) Helping patients who drink too much: a clinician’s guide. (2005) 2007 May 28; 2013.

Retrieved May 28, 2013, from. • National Institute on Drug Abuse (NIDA) Screening for Drug Use in Medical Settings. 2010 Retrieved May 28, 2013, from • Neumann T, Neuner B, Gentilello LM, Weiss-Gerlach E, Mentz H, Rettig JS, Spies CD. Gender differences in the performance of a computerized version of the alcohol use disorders identification test in subcritically injured patients who are admitted to the emergency department. Alcohol Clin Exp Res. 2004; 28(11):1693–1701.

[] • New York City Department of Health and Mental Hygiene (DOHMH) Trends in cigarette use among adults in New York City, 2002–2010. • Proctor SL, Hoffmann NG. A brief alternative for identifying alcohol use disorders. Subst Use Misuse. 2012; 47(7):847–860. Doi: 10.314.2012.674166.

[] [] • Reichmann William M, Losina Elena, Seage George R, Arbelaez Christian, Safren Steven A, Katz Jeffrey N, Walensky Rochelle P. Does modality of survey administration impact data quality: audio computer assisted self interview (ACASI) versus self-administered pen and paper? 2010; 5(1) [] [] • Reinert DF, Allen JP. The Alcohol Use Disorders Identification Test (AUDIT): a review of recent research. Alcohol Clin Exp Res. 2002; 26(2):272–279.

[] • Robert Wood Johnson Foundation and Schneider Institute for Health Policy. Substance abuse: The nation’s number one health problem.

• Rogers SM, Willis G, Al-Tayyib A, Villarroel MA, Turner CF, Ganapathi L, Jadack R. Audio computer assisted interviewing to measure HIV risk behaviours in a clinic population. Sex Transm Infect.

2005; 81(6):501–507. Doi: 10.1136/sti.2004.014266. [] [] [] • Saitz RP, Alford DJ, Bernstein DM, Cheng J, Samet, Palfai T.

Screening and brief intervention for unhealthy drug use in primary care settings: randomized clinical trials are needed. J Addict Med. 2010; 4(3):123–130. [] [] • Saitz R, Mulvey KP, Plough A, Samet JH. Physician unawareness of serious substance abuse. Am J Drug Alcohol Abuse. 1997; 23(3):343–354.

[] • Satre D, Wolfe W, Eisendrath S, Weisner C. Computerized screening for alcohol and drug use among adults seeking outpatient psychiatric services. Psychiatr Serv. 2008; 59(4):441–444. Doi: 10.1176/appi.ps.59.4.441. [] [] [] • Schackman Bruce R, Dastur Zubin, Rubin David S, Berger Judith, Camhi Eli, Netherland Julie, Finkelstein Ruth. Feasibility of using audio computer-assisted self-interview (ACASI) screening in routine HIV care.

2009; 21(8):992–999. [] • Selin KH. Test-retest reliability of the alcohol use disorder identification test in a general population sample. Alcohol Clin Exp Res. 2003; 27(9):1428–1435. Doi: 10.1097/01.alc.23230.4a. [] [] • Smith Peter C, Schmidt Susan M, Allensworth-Davies Donald, Saitz Richard.

A single-question screening test for drug use in primary care. Arch Intern Med. 2010; 170(13):1155–1160. [] [] • Sobell LC, Kwan E, Sobell MB. Reliability of a drug history questionnaire (DHQ) Addict Behav. 1995; 20(2):233–241.

[] • Solberg Leif I, Maciosek Michael V, Edwards Nichol M. Primary care intervention to reduce alcohol misuse ranking its health impact and cost effectiveness. Am J Prev Med. 2008; 34(2):143–152. [] • Spear S, Schroeder S, Gong-Guy E, Rawson R.

Substance abuse screening in the 21st Century: A pilot study of a computerized, self-administered ASSIST. Paper presented at the American Society of Addiction Medicine 40th Annual Medical-Scientific Conference; New Orleans. • Spitznagel EL, Helzer JE. A Proposed Solution to the Base Rate Problem in the Kappa-Statistic. Archives of General Psychiatry. 1985; 42(7):725–728.

[] • Substance Abuse and Mental Health Services Administration (SAMHSA); C. Treatment, editor. A guide to substance abuse services for primary care clinicians. Rockville, MD: Substance Abuse and Mental Health Services Administration; 1997. • Substance Abuse and Mental Health Services Administration (SAMHSA) White paper: Screening, brief intervention and referral to treatment in behavioral healthcare.

2011 Retrieved May 28, 2013, from. • Substance Abuse and Mental Health Services Administration (SAMHSA). Results from the 2011 National Survey on Drug Use and Health: National Findings. Office of Applied Studies (HHS Publication No. SMA 12–4713) 2012 Retrieved May 28, 2013, from. • Tai B, McLellan AT.

Integrating information on substance use disorders into electronic health record systems. J Subst Abuse Treat. 2012; 43(1):12–19. [] [] • Thompson WD, Walter SD. A reappraisal of the kappa coefficient. J Clin Epidemiol. 1988; 41(10):949–958.

[] • Tourangeau R, Smith TW. Asking sensitive questions - The impact of data collection mode, question format, and question context. Public Opinion Quarterly.

1996; 60(2):275–304. Doi: 10.1086/297751. [] • Vaca F, Winn D, Anderson C, Kim D, Arcila M. Feasibility of emergency department bilingual computerized alcohol screening, brief intervention, and referral to treatment. 2010; 31(4):264–269. Doi: 10.107.2010.514245. [] [] • Whitlock Evelyn P, Polen Michael R, Green Carla A, Orleans Tracy, Klein Jonathan.

Behavioral counseling interventions in primary care to reduce risky/harmful alcohol use by adults: a summary of the evidence for the U.S. Preventive Services Task Force.

Ann Intern Med. 2004; 140(7):557–568. [] • Wight RG, Rotheram-Borus MJ, Klosinski L, Ramos B, Calabro M, Smith R. Screening for transmission behaviors among HIV-infected adults. Aids Education and Prevention. 2000; 12(5):431–441.

[] • Yudko E, Lozhkina O, Fouts A. A comprehensive review of the psychometric properties of the Drug Abuse Screening Test. J Subst Abuse Treat. 2007; 32(2):189–198. Doi: 10.1016/j.jsat.2006.08.002.