Precision and linearity targets for validation of an IFNγ ELISPOT, cytokine flow cytometry, and tetramer assay using CMV peptides
© Maecker et al; licensee BioMed Central Ltd. 2008
Received: 06 April 2007
Accepted: 17 March 2008
Published: 17 March 2008
Single-cell assays of immune function are increasingly used to monitor T cell responses in immunotherapy clinical trials. Standardization and validation of such assays are therefore important to interpretation of the clinical trial data. Here we assess the levels of intra-assay, inter-assay, and inter-operator precision, as well as linearity, of CD8+ T cell IFNγ-based ELISPOT and cytokine flow cytometry (CFC), as well as tetramer assays.
Precision was measured in cryopreserved PBMC with a low, medium, or high response level to a CMV pp65 peptide or peptide mixture. Intra-assay precision was assessed using 6 replicates per assay; inter-assay precision was assessed by performing 8 assays on different days; and inter-operator precision was assessed using 3 different operators working on the same day. Percent CV values ranged from 4% to 133% depending upon the assay and response level. Linearity was measured by diluting PBMC from a high responder into PBMC from a non-responder, and yielded R2 values from 0.85 to 0.99 depending upon the assay and antigen.
These data provide target values for precision and linearity of single-cell assays for those wishing to validate these assays in their own laboratories. They also allow for comparison of the precision and linearity of ELISPOT, CFC, and tetramer across a range of response levels. There was a trend toward tetramer assays showing the highest precision, followed closely by CFC, and then ELISPOT; while all three assays had similar linearity. These findings are contingent upon the use of optimized protocols for each assay.
Validation of immunological assays can take a number of forms, and is required for compliance with Good Laboratory Practice (GLP), or for submission of data to licensing agencies. Two basic components of assay validation are the demonstration that an assay performs with adequate reproducibility for the intended purpose, and that the assay readout is linear over a useful range of data .
Specific guidelines exist for validation of traditional immunoassays such as ELISA, and the expected levels of precision and linearity of these assays are well-known [2, 3]. Less well-characterized are cellular immunity assays, of which the single-cell assays like tetramer staining , cytokine flow cytometry (CFC) [5, 6], and ELISPOT  are among the most popular. Some data has been published regarding the precision of individual assays [8–10], and there is very limited data on the linearity of CFC . However, precision and linearity have not been compared across assays, and expected levels of precision and linearity of these assays have not been determined in a side-by-side fashion.
Precision and linearity are important aspects of cellular immunity assays, since (a) cellular assays are inherently more complex, and thus less reproducible, than traditional immunoassays; and (b) cellular immunity assays are frequently used to detect rare antigen-specific T cell populations, which may be present at or near the assay detection limit. It is thus crucial to demonstrate that an assay is reproducible enough to generate reliable data in the response range expected for, say, a vaccine clinical trial, and that linearity is adequate to quantitatively compare results between treatment groups or between trials.
Fortunately, we now know that at least some vaccines to HIV and cancer, for example, can generate readily detectable T cell responses by assays such as tetramer, CFC, and ELISPOT [12, 13]. Still, there is wide variability in the performance of such assays between labs . Compliance with GLP thus requires that a given lab demonstrate its proficiency for a given assay, preferably with reference to an accepted standard.
Here we compare results from optimized protocols for tetramer staining, CFC, and ELISPOT, performed on shared cryopreserved PBMC specimens, with expert laboratories performing the individual assays. From this data, we derive target values for those who wish to determine precision and linearity of these assays in their own laboratory, and we also facilitate comparison of the three assays with regard to their relative precision and linearity.
Study design and response levels of donors
Mean response levels of the three CMV-responsive donors
% CD3+CD8+ cells
ELISPOT (peptide mix)
CFC (peptide mix)
Donor 41 (low)
270 SFC1 [0.53%]
8 SFC1 [0.02%]
Donor 68 (med)
285 SFC1 [0.94%]
31 SFC1 [0.10%]
Donor 43 (high)
95 SFC1 [0.14%]
28 SFC1 [0.04%]
Precision of individual assays
Comparative precision of the three assays
A number of studies have been published comparing ELISPOT, CFC, and tetramer staining [15, 18–27]. However, while these studies have addressed issues such as relative sensitivity of the assays, direct comparisons of precision and linearity have not been done. The best previous assessments of inter-laboratory precision came from standardization studies carried out for ELISPOT [14, 28] or CFC . The present study was designed to compare these assays with regard to precision and linearity, and in so doing, to provide target values for laboratories wishing to validate their own protocols for any of these assays. It should be noted that these targets may be specific to IFNγ and to the antigen systems used (CMV pp65495–503 and pp65 peptde mix), and can not necessarily be generalized to other cytokines and antigens.
In comparing assays, the choice of protocol is crucial; one wishes to compare fully-optimized versions of each assay. As such, we chose individual labs recognized as experts in each type of assay (tetramer, CFC, and ELISPOT) and allowed them each to use their own optimized protocol. In the case of ELISPOT, this protocol did not use costimulatory antibodies, while in CFC it did. Validation data in the lab doing ELISPOT showed that inclusion of costimulatory antibodies was not warranted based on increased backgrounds (data not shown); while similar data in the CFC lab supported the use of costimulation. The net result, that the assays might not give quantitatively equivalent results, was accepted in favour of the idea that each assay was optimally configured for its own best performance.
Often, CV has been used as a measure of precision that is independent of the units in which data is expressed (e.g., SFC or percent positive cells) . However, the stability of the CV deteriorates at very low means (Figure 1), so it is difficult to use it as a robust measurement of precision for low-level responses. Since antigen-specific assays are often used to assess very low response values, we chose instead to focus on SD, which we found to be highly linear with the response mean (Figure 2).
In general, there was a trend toward higher precision in intra-assay measurements compared to inter-assay or inter-operator measurements, in agreement with a previous study on CFC . Since the inter-assay and inter-operator precision studies were not repeated multiple times, we could not derive statistical tolerances around the SD values in Figure 2. However, a laboratory wishing to validate these assays could use this figure as a guide, comparing their own internal validation data to the figure for any given response level.
In order to compare precision across all three assays, ELISPOT values were converted to the same scale as CFC and tetramer staining (Figure 3), and the SD was plotted versus mean. Deriving a regression line through this data for each assay, we observed that ELISPOT SD tended to be higher, for any given response level, than CFC or tetramer SD. While we did not validate this trend statistically, it is concordant with previous studies that reported relatively low inter-laboratory precision for ELISPOT  and relatively high inter-laboratory precision for CFC , at least when data were analyzed centrally.
Why would ELISPOT assays have poorer precision in this study? One relevant factor could be the number of cells collected per data point, since assay precision is dependent upon counting statistics. The ELISPOT assay was optimized such that 2 × 105 PBMC per well gave the highest counting efficiency. CFC assays collected 40,000 CD3+CD8+ cells per sample, and tetramer assays collected 30,000 CD8+ T cells. Assuming CD3+CD8+ cells to be about 15% of PBMC, this means that roughly 30,000 CD3+CD8+ cells were present in each ELISPOT well. While the difference relative to CFC (30,000 versus 40,000) could explain some difference in precision, it is unlikely to be a major factor, at least at the higher response levels tested.
Another factor in ELISPOT assays is the fact that precision is dependent upon cell counting and pipetting. All calculation is based upon the assumption that 2 × 105 PBMC were in fact plated in every well. If the original PBMC count were in error, this would introduce a systematic bias in that assay's results that would potentially affect inter-assay or inter-operator reproducibility. And if pipetting were not precise, intra-assay reproducibility would be affected as well. In contrast, tetramer and CFC assays have a percentage readout that is less dependent upon the true number of cells plated per sample.
The ELISPOT assay is also unique in that it is a monolayer assay, and cell stimulation could be limited by the ability of T cells and APC to interact when they are spread out over a filter-bottom well. This could affect the reproducibility of stimulation, and could also explain undercounting of responsive T cells in ELISPOT relative to CFC. Lack of costimulatory antibodies in the ELISPOT assay, which were used in CFC, could also cause undercounting. Fnally, undercounting could result from coincident spots or other imprecisions in the computer-aided ELISPOT counting system. We saw such undercounting to a variable degree in this study, when ELISPOT results were converted to the same scale as CFC and tetramer results (see Table 1). ELISPOT has been shown in other studies to undercount positive events by 4- to 10-fold relative to CFC or tetramer staining [23, 24], even in the same CMV system used for the current study . Thus, the ELISPOT regression line for SD versus mean (Figure 3) could be expected to shift leftward relative to CFC and tetramer, if SD for a given sample were actually equal between the three assays.
For all the above reasons, it is not surprising that ELISPOT precision was not as high as CFC and tetramer. In fact, common practice in the field takes this into account, in that ELISPOT assays are typically done with 3–6 replicates per sample, while this is rarely done in CFC and tetramer assays. It was only for purposes of fair comparison that a single well was considered as one data point in this study.
With regard to linearity, we used a dilution methodology in which responder PBMC were diluted into non-responder PBMC. There was the potential for confounding artifacts with this method, including the possibility of loss of relevant antigen-presenting cells with serial dilution, and the possibility of allogeneic responses. Fortunately, these were not observed as significant factors. Linearity of CFC and ELISPOT were similar to tetramer, except in the instance where ELISPOT response was disproportionately low compared to the CFC or tetramer response. This suggests that antigen-presenting cell frequency was not a major limiting factor. In terms of allogeneic responses, unstimulated CFC background at each dilution was 0–0.07% of CD8+ T cells, which was not different from background in the assays that did not contain allogeneic cells. For ELISPOT, the average background in the linearity studies was 4.7 SFC per 2.5 × 105 PBMC, compared to 5.0 SFC when donor 43 (high responder) PBMC were tested alone.
In terms of assay comparison, it is worth noting that the ELISPOT assay does not easily allow for phenotyping of responsive cells as CD4+ or CD8+, etc. In our study, we fortuitously chose donors whose pp65 peptide mix responses were almost entirely CD8-restricted (pp65495–503 peptide responses should be CD8-restricted by nature). We thus focused on CD8+ T cell responses for all CFC assays. However, when using either CMV pp65495–503 or peptide mix stimulation, we observed that the rank order of donors was different for ELISPOT compared to the other two assays. CD4 responses in CFC were 0.2–0.4% for all donors (data not shown), so this was not sufficient to account for the difference. One outcome of this anomaly is that the pp65495–503 linearity data for ELISPOT was generated with a donor whose ELISPOT response was many fold lower than that donor's CFC or tetramer response. We thus focused more attention on the pp65 peptide mix ELISPOT response, which was at least somewhat higher, and which showed comparable linearity to CFC and tetramer.
Tetramer staining is by far the simplest of the three assays, requiring only phenotypic staining of PBMC. It is therefore not unexpected that it would have the highest precision, as the complexity of activation and sample processing should negatively impact precision of CFC and ELISPOT. However, it is interesting that CFC precision and linearity were not much different from that of tetramer staining. This is consistent with the notion that precision of CFC assays is largely related to gating and analysis, rather than activation or processing variables .
Although our focus in this paper is on precision and linearity, a limit of detection for each assay can also be determined from our data. This limit could be defined, for example, as 2 SD above the mean of replicate negative control samples. Such a limit is of course dependent upon the number of events collected, as previously described for CFC , as well as the negative control background, which varies between donors. In our study, we calculated an average lower limit of detection of 0.08% for CD8+ T cell IFNγ production in CFC, and 64 SFC per 2.5 × 105 PBMC in IFNγ ELISPOT. Note that these detection limits were the average across three donors, and were lower for those donors with low background. A similar calculation could not be done for tetramer staining, because a negative control (irrelevant tetramer) was not used.
From this comparison study, we provide target values for precision and linearity of tetramer, CFC, and ELISPOT assays, using cryopreserved PBMC. We conclude that all three assays can be performed with reasonable precision and linearity. Intra-assay precision was generally lower than inter-assay or inter-operator precision. Tetramer staining tended to have the highest precision and linearity, followed closely by CFC and then ELISPOT.
Donors and CMV responses
Three HLA-A2+, CMV seropositive healthy subjects were chosen from previous work, which suggested that they represented low (~0.1% tetramer+ or IFNγ+ cells, or ~30 SFC per 2.5 × 105 PBMC), medium (~0.3% tetramer+ or IFNγ+ cells, or ~100 SFC per 2.5 × 105 PBMC), and high (~1% tetramer+ or IFNγ+ cells, or ~200 SFC per 2.5 × 105 PBMC) responders to CMV pp65495–503 peptide. Actual mean values obtained in this study are shown in Table 1.
Collection and cryopreservation of PBMC
PBMC from leukapheresis were isolated using Ficoll gradient separation as previously described . To cryopreserve PBMC, 2× freezing media was first prepared, containing 20% DMSO in RPMI (Sigma Chemical Co., St. Louis, MO) containing 12.5% human serum albumin (HSA) (Gemini Bioproducts, Woodland, CA), and cooled on ice for a minimum of 30 minutes. Ficolled PBMC at 2 × 107 viable lymphocytes/ml were resuspended in cold RPMI+12.5% HSA with no DMSO. An equal volume of chilled 2× freezing media was added to the cell suspension dropwise, while gently swirling the tube. One ml of this cell suspension was aliquoted into each cryovial (Sarstedt, Inc., Newton, NC). Once aliquoted, cryovials were placed on ice and then transferred into a freezing container (Nalgene, Rochester, NY), and stored at -80°C for 24 hours. Cryovials were then transferred into liquid nitrogen for long-term storage. After 30 days, cryovials were overnight shipped on dry ice to the recipient laboratories.
Cryopreserved PBMC were stored at -80°C until thawing to set up the assays. Cryopreserved cells were thawed and slowly diluted with 8 ml of warm RPMI+10% fetal bovine serum+antibiotics (cRPMI-10, all components from Sigma). The cells were centrifuged for 7 minutes at 250 × G, then resuspended as described below for each assay. Viability and recovery were checked using Trypan blue, and were > 80% and > 50%, respectively, in all samples.
A common source of peptide antigens was shared among laboratories for CFC and ELISPOT assays. These included CMV pp65 peptide mix (BD Biosciences, San Jose, CA; used at a final concentration of 1.7 μg/ml/peptide); and CMV pp65495–503 peptide (SynPep Corp., Dublin, CA; used at a final concentration of 10 μg/ml).
Staining was done using the Multiple Antibody Single Color protocol (iMASC, Beckman Coulter Inc., Fullerton, CA) as previously described . Briefly, PBMC were resuspended at 1 × 107 per mL in HBSS+0.1% bovine serum albumin+0.02% sodium azide. 100 μL of PBMC per sample were stained for 30 minutes at room temperature using CD8 FITC, HLA-A2 tetramer loaded with CMV pp65495–503 PE, and CD4, CD13, and CD19 PE-Cy5 (all from Beckman Coulter). Samples were washed and analyzed on a FACS Calibur flow cytometer (BD Biosciences). 30,000 CD8+ T cells were collected, and results were reported as the percentage of CD8+, PE-Cy5-negative cells that were tetramer-positive. No correction for background staining with irrelevant tetramer was used, but the test tetramer showed undetectable staining on CMV-negative donors (not shown).
Activation and processing were done as previously described . Briefly, PBMC were resuspended at 1 × 107 per mL in cRPMI-10, and 200 μL were plated per well in 96-well round-bottom plates. After overnight rest at 37°C, activation reagents (stimulus+brefeldin A+costimulatory antibodies to CD28 and CD49d) were added and the cells incubated at 37°C for 6 hours. The cells were then fixed and permeabilized as per reference , followed by staining with IFNγ FITC/CD69 PE/CD8 PerCPCy5.5/CD3 APC (BD Biosciences) for 1 hour at room temperature in dark. Plates were washed and cells resuspended in 1% paraformaldehyde in PBS, then acquired on a FACSCalibur flow cytometer (BD Biosciences). 40,000 CD3+CD8+ lymphocytes were collected per sample. Data were reported as the net percent of CD3+CD8+lymphocytes that were CD69+IFNγ+ after subtracting the average response of unstimulated samples.
Activation and processing were done largely as previously described . Briefly, Multiscreen-HA 96-well plates (Millipore, Bedford, MA) were coated with mouse anti-human IFNγ mAb, and the plates washed and blocked as per the referenced publication. PBMC were resusupended at 1 × 106 per mL, and 200 μL plated per well along with the appropriate antigen. Stimulation was for 18–24 hours at 37°C in 5% CO2. Plates were washed and developed as in reference , and the number of spots per well was determined using a KS ELISPOT Automated Reader System with KS ELISPOT 4.2 Software (Carl Zeiss, Inc., Thornwood, NY). SFC per 2.5 × 105 PBMC were reported after subtracting the average response of unstimulated samples.
Design of precision and linearity studies
Data for each assay were acquired on a single instrument, using the same settings throughout the study. For flow cytometry-based assays, an optimized gating template and instrument settings file were used throughout the study, including inter-operator precision studies. All data for linearity, intra-assay, and inter-assay precision were performed by a single operator. For conversion of ELISPOT data to percent of CD3+CD8+ cells, the percent of CD3+CD8+ cells was determined from a representative subset of the CFC data, and was found to be very consistent across days and across operators. However, it varied significantly from one donor to the next (see Table 1). The ELISPOT data was converted to percent response as follows: SFC/2.5 × 105/(mean CD3+CD8+ percent for that donor)×100.
For comparison of SD among samples in intra-assay versus inter-assay or inter-operator studies, a Wilcoxon Matched Pairs test was used. Statistical analyses were carried out in GraphPad Prism software (San Diego, CA).
This work was supported by grant #U54 CA090818 from the National Cancer Institute, National Institutes of Health.
- Owens MA, Vall HG, Hurley AA, Wormsley SB: Validation and quality control of immunophenotyping in clinical flow cytometry. J Immunol Methods. 2000, 243: 33-50. 10.1016/S0022-1759(00)00226-X.View ArticlePubMedGoogle Scholar
- Borg L, Kristiansen J, Christensen JM, Jepsen KF, Poulsen LK: Evaluation of accuracy and uncertainty of ELISA assays for the determination of interleukin-4, interleukin-5, interferon-gamma and tumor necrosis factor-alpha. Clin Chem Lab Med. 2002, 40: 509-19. 10.1515/CCLM.2002.089.View ArticlePubMedGoogle Scholar
- Findlay JW, Smith WC, Lee JW, Nordblom GD, Das I, DeSilva BS, Khan MN, Bowsher RR: Validation of immunoassays for bioanalysis: a pharmaceutical industry perspective. J Pharm Biomed Anal. 2000, 21: 1249-73. 10.1016/S0731-7085(99)00244-7.View ArticlePubMedGoogle Scholar
- Altman JD, Moss PAH, Goulder PJR, Barouch DH, McHeyzer-Williams MG, Bell JI, McMichael AJ, Davis MM: Phenotypic analysis of antigen-specific T lymphocytes. Science. 1996, 274: 94-6. 10.1126/science.274.5284.94.View ArticlePubMedGoogle Scholar
- Suni MA, Picker LJ, Maino VC: Detection of antigen-specific T cell cytokine expression in whole blood by flow cytometry. J Immunol Methods. 1998, 212: 89-98. 10.1016/S0022-1759(98)00004-0.View ArticlePubMedGoogle Scholar
- Waldrop SL, Pitcher CJ, Peterson DM, Maino VC, Picker LJ: Determination of antigen-specific memory/effector CD4+ T cell frequencies by flow cytometry: evidence for a novel, antigen-specific homeostatic mechanism in HIV-associated immunodeficiency. J Clin Invest. 1997, 99: 1739-50. 10.1172/JCI119338.PubMed CentralView ArticlePubMedGoogle Scholar
- Czerkinsky C, Andersson G, Ekre HP, Nilsson LA, Klareskog L, Ouchterlony O: Reverse ELISPOT assay for clonal analysis of cytokine production. I. Enumeration of gamma-interferon-secreting cells. J Immunol Methods. 1988, 110: 29-36. 10.1016/0022-1759(88)90079-8.View ArticlePubMedGoogle Scholar
- Lathey J: Preliminary steps toward validating a clinical bioassay: case study of the ELISpot assay. Biopharm Intl. 2003, 42-50.Google Scholar
- Lathey J, Sathiyaseelan J, Matijevic M, Hedley ML: Validation of pretrial ELISspot measurements. BioProcess Intl. 2003, 34-41.Google Scholar
- Maecker HT, Rinfret A, D'Souza P, Darden J, Roig E, Landry C, Hayes P, Birungi J, Anzala O, Garcia M, et al.: Standardization of cytokine flow cytometry assays. BMC Immunol. 2005, 6: 13-10.1186/1471-2172-6-13.PubMed CentralView ArticlePubMedGoogle Scholar
- Nomura LE, Walker JM, Maecker HT: Optimization of whole blood antigen-specific cytokine assays for CD4(+) T cells. Cytometry. 2000, 40: 60-68. 10.1002/(SICI)1097-0320(20000501)40:1<60::AID-CYTO8>3.0.CO;2-J.View ArticlePubMedGoogle Scholar
- Maino VC, Suni MA, Wormsley SB, Carlo DJ, Wallace MR, Moss RB: Enhancement of HIV type 1 antigen-specific CD4+ T cell memory in subjects with chronic HIV type 1 infection receiving an HIV type 1 immunogen. AIDS Res Hum Retroviruses. 2000, 16: 539-47. 10.1089/088922200308954.View ArticlePubMedGoogle Scholar
- Smith JW, Walker EB, Fox BA, Haley D, Wisner KP, Doran T, Fisher B, Justice L, Wood W, Vetto J, et al.: Adjuvant immunization of HLA-A2-positive melanoma patients with a modified gp100 peptide induces peptide-specific CD8+ T-cell responses. J Clin Oncol. 2003, 21: 1562-73. 10.1200/JCO.2003.09.020.View ArticlePubMedGoogle Scholar
- Cox JH, Ferrari G, Kalams SA, Lopaczynski W, Oden N, D'Souza M P: Results of an ELISPOT proficiency panel conducted in 11 laboratories participating in international human immunodeficiency virus type 1 vaccine trials. AIDS Res Hum Retroviruses. 2005, 21: 68-81. 10.1089/aid.2005.21.68.View ArticlePubMedGoogle Scholar
- Maecker HT, Moon J, Bhatia S, Ghanekar SA, Maino VC, Payne JK, Kuus-Reichel K, Chang JC, Summers A, Clay TM, et al.: Impact of cryopreservation on tetramer, cytokine flow cytometry, and ELISPOT. BMC Immunol. 2005, 6: 17-10.1186/1471-2172-6-17.PubMed CentralView ArticlePubMedGoogle Scholar
- Kern F, Surel IP, Brock C, Freistedt B, Radtke H, Scheffold A, Blasczyk R, Reinke P, Schneider-Mergener J, Radbruch A, et al.: T-cell epitope mapping by flow cytometry. Nat Med. 1998, 4: 975-8. 10.1038/nm0898-975.View ArticlePubMedGoogle Scholar
- Kern F, Bunde T, Faulhaber N, Kiecker F, Khatamzas E, Rudawski IM, Pruss A, Gratama JW, Volkmer-Engert R, Ewert R, et al.: Cytomegalovirus (CMV) phosphoprotein 65 makes a large contribution to shaping the T cell repertoire in CMV-exposed individuals. J Infect Dis. 2002, 185: 1709-16. 10.1086/340637.View ArticlePubMedGoogle Scholar
- Asemissen AM, Nagorsen D, Keilholz U, Letsch A, Schmittel A, Thiel E, Scheibenbogen C: Flow cytometric determination of intracellular or secreted IFNgamma for the quantification of antigen reactive T cells. J Immunol Methods. 2001, 251: 101-8. 10.1016/S0022-1759(01)00312-X.View ArticlePubMedGoogle Scholar
- Heeger PS, Greenspan NS, Kuhlenschmidt S, Dejelo C, Hricik DE, Schulak JA, Tary-Lehmann M: Pretransplant frequency of donor-specific, IFN-gamma-producing lymphocytes is a manifestation of immunologic memory and correlates with the risk of posttransplant rejection episodes. J Immunol. 1999, 163: 2267-75.PubMedGoogle Scholar
- Helms T, Boehm BO, Asaad RJ, Trezza RP, Lehmann PV, Tary-Lehmann M: Direct visualization of cytokine-producing recall antigen-specific CD4 memory T cells in healthy individuals and HIV patients. J Immunol. 2000, 164: 3723-32.View ArticlePubMedGoogle Scholar
- Hobeika AC, Morse MA, Osada T, Ghanayem M, Niedzwiecki D, Barrier R, Lyerly HK, Clay TM: Enumerating antigen-specific T-cell responses in peripheral blood: a comparison of peptide MHC Tetramer, ELISpot, and intracellular cytokine analysis. J Immunother. 2005, 28: 63-72. 10.1097/00002371-200501000-00008.View ArticlePubMedGoogle Scholar
- Karlsson AC, Martin JN, Younger SR, Bredt BM, Epling L, Ronquillo R, Varma A, Deeks SG, McCune JM, Nixon DF, et al.: Comparison of the ELISPOT and cytokine flow cytometry assays for the enumeration of antigen-specific T cells. J Immunol Methods. 2003, 283: 141-53. 10.1016/j.jim.2003.09.001.View ArticlePubMedGoogle Scholar
- Kuzushima K, Hoshino Y, Fujii K, Yokoyama N, Fujita M, Kiyono T, Kimura H, Morishima T, Morishima Y, Tsurumi T: Rapid determination of Epstein-Barr virus-specific CD8(+) T-cell frequencies by flow cytometry. Blood. 1999, 94: 3094-100.PubMedGoogle Scholar
- Moretto WJ, Drohan LA, Nixon DF: Rapid quantification of SIV-specific CD8 T cell responses with recombinant vaccinia virus ELISPOT or cytokine flow cytometry. Aids. 2000, 14: 2625-7. 10.1097/00002030-200011100-00034.View ArticlePubMedGoogle Scholar
- Pahar B, Li J, Rourke T, Miller CJ, McChesney MB: Detection of antigen-specific T cell interferon gamma expression by ELISPOT and cytokine flow cytometry assays in rhesus macaques. J Immunol Methods. 2003, 282: 103-15. 10.1016/j.jim.2003.08.003.View ArticlePubMedGoogle Scholar
- Sun Y, Iglesias E, Samri A, Kamkamidze G, Decoville T, Carcelain G, Autran B: A systematic comparison of methods to measure HIV-1 specific CD8 T cells. J Immunol Methods. 2003, 272: 23-34. 10.1016/S0022-1759(02)00328-9.View ArticlePubMedGoogle Scholar
- Whiteside TL, Zhao Y, Tsukishiro T, Elder EM, Gooding W, Baar J: Enzyme-linked immunospot, cytokine flow cytometry, and tetramers in the detection of T-cell responses to a dendritic cell-based multipeptide vaccine in patients with melanoma. Clin Cancer Res. 2003, 9: 641-9.PubMedGoogle Scholar
- Samri A, Durier C, Urrutia A, Sanchez I, Gahery-Segard H, Imbart S, Sinet M, Tartour E, Aboulker JP, Autran B, et al.: Evaluation of the interlaboratory concordance in quantification of human immunodeficiency virus-specific T cells with a gamma interferon enzyme-linked immunospot assay. Clin Vaccine Immunol. 2006, 13: 684-97. 10.1128/CVI.00387-05.PubMed CentralView ArticlePubMedGoogle Scholar
- Motulsky H: Intuitive Biostatistics. 1995, Oxford: Oxford Univ. Press, 29-Google Scholar
- Maecker HT: The role of immune monitoring in evaluating cancer immunotherapy. Cancer Drug Discovery and Development: Immunotherapy of Cancer. Edited by: Disis ML. 2005, Totowa, NJ: Humana Press, 59-71.Google Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.