Optimization and evaluation of Luminex performance with supernatants of antigen-stimulated peripheral blood mononuclear cells
© The Author(s). 2016
Received: 6 June 2016
Accepted: 3 November 2016
Published: 11 November 2016
The Luminex bead-based multiplex assay is useful for quantifying immune mediators such as cytokines and chemokines. Cross-comparisons of reagents for this technique from different suppliers have already been performed using serum or plasma but rarely with supernatants collected from antigen-stimulated peripheral blood mononuclear cells (PBMC). Here, we first describe an optimization protocol for cell culture including quantity of cells and culture duration to obtain reproducible cytokine and chemokine quantifications. Then, we compared three different Luminex kit suppliers.
Intraclass correlation coefficients (ICCs) for a 2-days stimulation protocol were >0.8 for IFNγ and Perforin. The specific concentration was maximal after two or five days of stimulation, depending on the analyte, using 0.5 million PBMC per well, a cell quantity that gave the same level of specific cytokine secretion as 1.0 million. In the second part of the study, Luminex kits from Millipore showed a better working range than Bio-Rad and Ozyme ones. For tuberculin purified protein derivative (PPD)-stimulated samples, the overall mean pooled coefficients of variation (CVs) for all donors and all cytokines was 17.2 % for Bio-Rad, 19.4 % for Millipore and 26.7 % for Ozyme. Although the different kits gave cytokine concentrations that were generally compatible, there were discrepancies for particular cytokines. Finally, evaluation of precision and reproducibility of a 15-plex Millipore kit using a “home-made” internal control showed a mean intra-assay CV <13 % and an inter-assay CV <18 % for each cytokine concentration.
A protocol with a single round of stimulation but with two time points gave the best results for assaying different cytokines. Millipore kits appear to be slightly more sensitive than those from Bio-Rad and Ozyme. However, we conclude that the panel of analytes that need to be quantified should be the main determinant of kit selection. Using an internal control we demonstrated that a 15-plex magnetic Milliplex kit displayed good precision and reproducibility. Our findings should help optimize assays for evaluating immune responses during the course of disease or infection, or in response to vaccine or therapy.
KeywordsLuminex Cytokine Reproducibility Precision Culture supernatant
Cytokines and chemokines are soluble molecules playing key roles in the innate and adaptive immune responses. Quantifying these immune mediators has become increasingly important to obtain better descriptions of immune responses during the course of disease or infections, or to a vaccine, therapy and intervention; such detailed descriptions allow the identification of correlates of cure or protection. Various methods can be used to measure cytokines and chemokines, and enzyme-linked immunosorbent assay (ELISA) is still the most extensively employed . This monoplex antibody-based immunoassay has been used for more than fifty years but can be laborious, sample consuming and expensive when studying profiles of large numbers of cytokines and chemokines. Therefore, a number of new technologies allowing simultaneous quantitative measurements of multiple cytokines and chemokines in small sample volumes have recently been developed. These multiplex immunoassay systems can be divided into two classes: planar assays and suspension microsphere assays . The Multi-array Meso Scale Discovery (MSD) electro-chemiluminescence detection system (planar immunoassay) and Luminex xMAP technology (microsphere immunoassay) are considered to be the two most suitable platforms for biomarker analysis or quantification [3–5].
The growing use of Luminex technology has led to different kits being made available by several companies, and we decided to work with this multiplex platform. The Luminex system allows the analysis of up to 100 analytes in a single well. Relatively small volumes (25–50 μl) of serum, plasma, other body fluids, or cell culture supernatant can be assayed for cytokines and chemokines. Extensive data have been published validating the Luminex platform for detection of multiple analytes, by comparing this technique with ELISA [6–12]. Cross-comparisons between reagent suppliers showed variability in absolute cytokine concentration results in serum or plasma [13–15]. Moreover, a study comparing Luminex kits from three different companies, including World Health Organization (WHO) cytokine standards in addition to the standards provided with each kit, revealed striking differences in results for cytokine concentrations in serum .
There have been few studies comparing different Luminex kits using supernatants from antigen-stimulated cell culture, an approach used in vaccine research. One study compared Luminex kits from three different suppliers for the determination of cytokine levels in whole blood culture supernatant , and two studies compared cytokine and chemokine levels in supernatants from antigen specific-stimulated peripheral blood mononuclear cells (PBMC) as tested with different commercial Luminex kits [18, 19]. These studies revealed differences of performance (accuracy, recovery, reproducibility) between the suppliers tested. Defawe and colleagues published a study on optimization and validation of a Luminex protocol to assess cytokine and chemokine production by vaccine-specific cells to allow better characterization of immune responses . However, the optimization of the “cell culture step” described in their work could be improved. The aim of the present study was to optimize a cell culture protocol (including quantity of cells, culture duration, and positive control) to obtain reproducible cytokine and chemokine quantifications by Luminex technology in antigen-stimulated PBMC supernatants; the second aim was then to compare the performances of different Luminex kits.
Reproducibility of the Luminex assay
Description of Luminex kits
Type of beads
Number of analytes
Part of the study
BIIR + Biolegend
IFNγ, IL-5, IL-17 + Perforin
Milliplex MAP Human Cytokine (#SPR137)
IL-17, IL-21, IP-10, IFNγ, IL-10, IL-13, IL-1β, IL-2, IL-5, IL-6, TNFα
Kinetics of cytokine production + Quantity of stimulated PBMC
Milliplex MAP Human Cytokine (#SPR171)
IL-17, IL-21, IFNγ, IL-10, IL-13, IL-1β, IL-2, IL-5, IL-6, TNFα
Kinetics of cytokine production + Quantity of stimulated PBMC
Milliplex MAP Human Th17
IL-17A, IL-22, IFNγ, IL-10, IL-13, IL-1β, IL-2, IL-5, IL-6, TNFα
Kinetics of cytokine production + Quantity of stimulated PBMC
Procarta Immunoassays kit (Affymetrix)
GM-CSF, IFNγ, IL-10, IL-12p40, IL-13, IL-17A, IL-1β, IL-2, IL-22, IL-5, IL-6, IP-10, MCP-1, MIP-1α, MIP-1β, TNFα
Comparison of commercial Luminex Kits
Milliplex MAP Human Cytokine (#SPR158) + Milliplex MAP Human Cytokine/Chemokine
15 + 1
GM-CSF, IFNγ, IL-10, IL-13, IL-17A, IL-1β, IL-2, IL-22, IL-5, IL-6, IP-10, MCP-1, MIP-1α, MIP-1β, TNFα + IL-12p40
Comparison of commercial Luminex Kits
Bio-plex Pro assays + Bio-Plex Pro Human Th17 Cytokine assays
28 + 1
IL-1β, IL-1ra, IL-2, IL-4, IL-5, IL-6, IL-7, IL-8, IL-9, IL-10, IL-12p70, IL-13, IL-15, IL-17A, Eotaxin, Basic FGF, G-CSF, GM-CSF, IFNγ, IP-10, MCP-1, MIP-1α, MIP-1β, PDGF-BB, RANTES, TNFα, VEGF, and IL-12p40 + IL-22
Comparison of commercial Luminex Kits
Milliplex MAP Human Th17
GM-CSF, IFNγ, IL-10, IL-17 F, IL-13, IL-17A, IL-1β, IL-2, IL-22, IL-9, CCL20/MIP-3α, IL-5, IL-23, IL-27, and TNFα
Internal QC study
Intraclass correlation coefficients (ICCs) across HIV-stimulated samples
Conc in range (pg/ml)
Total meas. (n)
Total meas. (n)
Total meas. (n)
Kinetics of cytokine production
We performed kinetic studies over 7 days with samples from two healthy donors to determine optimal culture duration for quantification of weakly secreted cytokines in stimulated-PBMC supernatants. First, we found that the culture duration (without rIL-2 supplemented medium) should not be longer than 6 days to maintain cell viability above 70 % using non-stimulated or tuberculin purified protein derivative (PPD)-stimulated PBMC (Additional file 3).
Number of stimulated PBMC
Comparison of three commercial Luminex kits
Next, we compared Luminex kits supplied by different manufacturers (Millipore, Ozyme and Bio-Rad). We first analysed mean % CV from standard and samples coming from polystyrene bead based 10-plex and magnetic bead based 15-plex kits and found that they were lower with magnetic beads (3.1 and 15.5, respectively) compared to polystyrene ones (5.3 and 25.1, respectively), thus confirming the better precision of magnetic kits as previously described . In the next experiments, only Luminex kits using magnetic beads were selected and analytes were quantified at their optimal time point previously defined.
Working range and sensitivities
We then used intra-assay coefficients of variation (CVs) to compare the precision of the three kits with standards and samples. All kits performed well for standard intra-assay precision: the mean of standard intra-plate CVs for the 16 common cytokines was below 8 % for all kits (Additional file 4: Figure S3). As this work was initially performed to select the best Luminex kit for a TB/HIV study, PBMC from two healthy donors, one HIV-1-positive patient and one tuberculosis (TB)-positive patient were cultured in triplicate for 2 and 5 days without antigen (NS) or with PPD, early secreted antigenic target 6 kDa protein (ESAT-6) or SEB. The supernatant of each culture well was considered to be one sample for Luminex analysis. Not all the cytokines were included in the final analysis because their values were above the ULOQ (MCP-1, MIP-1α, MIP-1β and IL-6) or below the LLOQ (IL-12p40). Eleven common cytokines were thus analyzed: IP-10, IL-10, IL-1β and IL-2 after 2-day culture and IL-5, GM-CSF, IFNγ, IL-17A, IL-13, IL-22, and TNFα after 5-day culture. CVs of three culture replicates for these 11 cytokines were plotted against the mean concentration for NS, PPD, SEB and ESAT-6 stimulations. A characteristic non-linear relationship was observed for all kits (Additional file 5: Figure S4): the CV rose substantially as the mean value approached zero.
Internal QC for Luminex study
For the last experiment, we used the Millipore kit because: 1) its working range and sensitivity were better than the two other kits tested, 2) %CVs were as good as the other kits, 3) no cross-reactivity was observed, 4) it was the only kit with QCs provided, and 5) it contains most analytes that we were interested in.
Inter-assay variability of internal control
The Luminex platform is widely used to study immune responses but only few studies evaluated its reproducibility and/or compared kits from different suppliers using supernatants from stimulated PBMC [18–20]. This work was designed to evaluate reproducibility of cytokine quantification by Luminex assay in supernatants from antigen-stimulated PBMC and to improve this quantification by optimization of the culture protocol and eventually the choice a particular commercial kit. We used cryopreserved PBMC because such samples are typical of those used in clinical trials. Moreover, it has been shown that using validated standard operating procedures (SOPs) for isolation, cryopreservation and thawing of PBMC, CD4+ and CD8+ cells maintain full functionality in cytokine enzyme-linked immunospot (ELISpot) assays . We first tested two different stimulation protocols: a two-round stimulation (i.e. cytokine measurement on day 11, after a first stimulation on day 0 and a re-stimulation on day 9) in the presence of rIL-2 supplemented medium, and a one-round stimulation for 1 to 7 days without rIL-2 supplemented medium. The two-round stimulation protocol induced a lot of variability. However, this type of protocol could be valuable to investigate “on/off” signals if no precise quantification of secreted cytokines is required. We also observed some variability, although to a lesser extent, with the 48 h stimulation protocol. Like ELISpot assay, the observed variability with the Luminex assay, which is inherent to cell culture, probably resulted in part from variations in counting and pipetting, and low antigen-specific cell frequency . One way to minimize this variability is to perform multi-well stimulations and to pool supernatants before assaying them by the Luminex platform.
Another factor to take into consideration for cytokine quantifications in stimulated-PBMC supernatants is the cell viability. Defawe et al. claimed that a high cell viability limited the potential effect of dead cells on cytokine content measured by Luminex . However, we observed that having cell viability above 80 % on the day of stimulation (D0) was not sufficient to ensure low cell mortality during the culture. Indeed, a dose of 100 ng/ml SEB (10-fold lower than that used in the study by Defawe and colleagues) killed near 50 % of PBMC after 5 days of culture (data not shown). Consequently, the antigen dose needs to be adapted.
Our results suggest that two different time-points are required for optimal quantifications of different cytokines (IFNγ, IL-2, IL-5 and IL-17) in supernatants from antigen-stimulated PBMC. This is consistent with published results. Indeed, differences in the kinetics of IFNγ and IL-2 production have already been observed in whole blood cell cultures . The duration of in vitro stimulation of PBMC with recall antigens determines the subset distribution of IFNγ-producing cells, and 4 or 5 days of stimulation are necessary to have a predominance of IFNγ-secreting T cells . Lagrelius and colleagues reported that 3 and 7 days are the most suitable periods for stimulation to measure various cytokine levels in whole blood supernatant . Furthermore, our results validating the use of 0.5 million PBMC per well are in agreement with those obtained by others with cytomegalovirus (CMV) stimulation .
After setting up our protocol for cytokine measurements in supernatants coming from antigen-stimulated PBMC, we compared results obtained with Luminex magnetic kits coming from three different manufacturers. We observed a lower sensitivity than that reported by the vendor for some analytes for all kits but overall, we obtained a better working range with Millipore than Ozyme and Bio-Rad kits. Also, Djoba Siawaya and colleagues reported a comparison between commercial Luminex cytokine kits using whole blood supernatant and concluded that the LINCO (now Millipore) kit was the most sensitive for measuring concentrations of numerous recombinant cytokines in samples that had been spiked with serial dilutions of the standards provided by the manufacturer . Consistent with our results, Belabani and colleagues showed that a Bio-Rad magnetic Luminex kit was not as sensitive as suggested by the manufacturer for some cytokines . In their condensed performance-validation strategy for multiplex detection kits used in studies of human clinical samples recently published, these authors proposed a complete validation of Luminex assays. Among the critical points, we would also recommend 1) extending the standard curve to include lower values to check the working range and true sensitivity; 2) including LLOQs and ULOQs as validation samples to validate their concentration; 3) excluding values below the LLOQ or using specific statistical analyses that takes into account left-censoring ; 4) diluting and reanalyzing all samples with a concentration above the ULOQ; and 5) fixing the thresholds of %CV and %Recovery at 25 % (30 % for LLOQs) as acceptance criteria for samples, as defined by Lee and colleagues .
We then compared the precision of the three kits with standards and samples. It is important to note that we used here triplicate cultures in our Luminex assays (and not supernatant pooled from several culture wells) to estimate the variability cytokine assays results, and this could explain at least in part the high variability observed. As observed for ELISpot , but also tetramer assays and cytokine flow cytometry , the relationship between CV and mean response level is not linear. The stability of the CV is poor at very low mean values, and antigen-specific assays are often used to analyze data in a range where CV is non-linear. Consequently, Maecker and colleagues suggested using Standard Deviation (SD), which may be related linearly to the response mean, to compare the precision of different assays . However, we did not find a clear linear correlation between SD and mean concentration (Additional file 8), and therefore we used %CV to compare precision of the different kits. Our results confirmed that the lower the secretion, the higher the %CV and that IL-5, IL-13 and IL-17A were difficult to quantify accurately even at the optimal time point, whatever the kit used. Another group has compared Luminex kits for the measurement of cytokines in supernatants collected from antigen-stimulated PBMC: they report good intra-plate but poor inter-plate precision of the Bio-Rad 17-plex kit  and found that the Millipore magnetic kit gave the most reproducible results . In our hands, the good precision of Millipore kits was confirmed, as mean % CV of standards and samples from 2 different kits (10-plex versus 15-plex magnetic beads based kits) were similar.
We also used a “home-made” internal control to evaluate precision and reproducibility of Milliplex kits as suggested by De Jager and colleagues in their work with serum, plasma and whole blood culture supernatant to define the prerequisites for cytokine measurements in clinical trials with multiplex immunoassays . We showed that the Millipore Milliplex kits demonstrated a good precision and reproducibility in our hands. Similarly, in their study of optimization and qualification of a multiplex bead array to assess cytokine and chemokine production by vaccine-specific cells, Defawe and colleagues found CVs for the intra-assay and inter-day variability below 30 % for all but one analyte with a Millipore kit .
Our study showed variability in the measurement of absolute cytokine concentration in supernatants from antigen-stimulated PBMC with Luminex kits coming from different manufacturers. A previous study showing that a Bio-Rad kit was more sensitive than an Invitrogen kit revealed the potential for differences in absolute cytokine concentration reported by these two kits . Other studies also showed variability in the measurement of absolute cytokine concentrations in serum or plasma [13–16]. Possibly, the different capture and detection antibodies are the major causes of the differences in performance of multiplex kits from different manufacturers as well as the heterogeneity in analytes within each kit. Also, differences between the purified recombinant proteins used to generate the standard curves, and between assay buffers supplied by the manufacturers are likely to contribute to the divergent results, as well as cross-reactivities observed with the Ozyme kit. Thus, the main criterion to select a manufacturer should be the availability of the panel of analytes that needs to be quantified and mixing data coming from different kits for analyses should be banned.
To determine positivity of samples, Defawe and colleagues used an empirical method based on their experience using the assay and they aimed to control the false positive rate below 3 % . The first criterion was “minimum analyte concentration over background” determined for each analyte and the second criterion was “minimum fold increase over background concentration”. The minimum fold increase over background concentration criterion was set as three-fold for all analytes. We used the same second criterion in this study to calculate the SI, but the definition used for the first criterion is debatable because there is no consensus formula to determine the minimum analyte concentration over background.
Here, we demonstrate that a protocol with a single round of stimulation but with two time points (day-2 and day-5) gave the optimal quantifications for different cytokines using 0.5 million PBMC per well, a cell quantity that gave the same level of specific cytokine secretion as 1.0 million. Millipore kits appear to be slightly more sensitive than those from Bio-Rad and Ozyme. However, we conclude that the panel of analytes that need to be quantified should be the main determinant of kit selection and that the same kit must be used throughout any given study. We also think that it is important to use the same batch number throughout the study to allow results to be compared between plates. To ensure the validity of the results, we also recommend the use of an internal control. Using this type of sample, we demonstrated that a 15-plex magnetic Milliplex kit displayed good precision and reproducibility. Moreover, our study suggests that particular attention should be paid to cell viability during cell culture to ensure that high cell mortality does not perturb the nature of cytokines secreted. Finally, to limit the variability inherent to cell culture, we suggest the use of supernatant pooled from at least three culture replicates to avoid large variations in the cytokine concentrations observed. In view of the cost of the technique and the good reproducibility of the results obtained with Millipore QC and with our internal control (intra- and inter-plate %CVs), it is unclear whether supernatant replicates are beneficial if standard curves are performed in duplicate and are extended to include lower levels so as to confirm the working range. In summary, our findings should help optimize assays for quantifying immune mediators during the course of disease or infection, or in response to vaccine or therapy.
Blood sample processing
Whole blood specimens were collected from two healthy controls (from the Etablissement Français du Sang, Créteil, France), seven HIV-seropositive patients (from the Mondor Hospital cohort, Créteil, France) and one TB-infected patient (from St-Louis Hospital, Paris, France).. PBMC were isolated using LSM 1077 density gradient media (PAA), washed with PBS without calcium or magnesium (Life Technologies), frozen in fetal calf serum (FCS, PAA) containing 10 % DMSO (Sigma) and stored in liquid nitrogen. For each experiment, about 12-18x106 PBMC (one cryovial) were thawed in R-20 (RPMI 1640 + glutamax containing 100 U/ml penicillin and 0.1 mg/ml streptomycin (Life Technologies) and 20 % FCS) medium and rested for 3 h at 37 °C, under 5 % CO2 in R-10HS (RPMI 1640 + glutamax containing 100U/ml penicillin and 0.1 mg/ml streptomycin and 10 % AB Human serum (HS, PAA)) medium. A minimal cell viability of 75 % as measured with an Auto 2000 cellometer (Nexcelom) after resting was required for use.
Two different protocols were used for PBMC stimulations. For the first one (2-step stimulation), PBMC (1×106) were stimulated for 2 days at 37 °C, under 5 % CO2 in a final volume of 300 μl R-10HS in 96 deep well plates (Greiner) with (i) 2 μg/ml of LIPO-5 vaccine composed of the clade B LAI HIV-1 Nef 66–97, Nef 116–145, Gag 17–35, Gag 253–284 and Pol 325–355 regions, coupled to a lipid tail ; (ii) 2 μg/ml of a mix of five long HIV-1 peptides (LP mix, NeoMPS) corresponding to the sequences included in the LIPO-5 vaccine without the lipid tail; (iii) 2 μg/ml of a pool of 36 HIV-1 15-mer peptides (NeoMPS) overlapping by 11 amino acids and spanning Nef, Gag and Pol sequences encoded by the LIPO-5 vaccine. Aliquots of 200 μl of supernatant were harvested on day 2, and 900 μl of fresh R-10HS medium containing 100 IU/ml human recombinant IL-2 (rIL-2, Miltenyi) was added. On days 4, 7 and 8, 500 μl of medium was replaced with fresh R10-HS medium containing 100 IU/ml rIL-2. On day 9, 800 μl of supernatant was discarded, cells were transferred in 96 V-bottom wells culture plates (Greiner), washed in PBS, resuspended in R-10HS medium containing the same concentrations of the corresponding antigens and incubated for 2 days at 37 °C, under 5 % CO2 in a final volume of 200 μl R-10HS. Positive and negative controls consisted of PBMC cultured with 100 ng/ml of SEB (Sigma, only for the two last days of stimulation) and PBMC cultured in R-10HS alone (non-stimulated, NS), respectively. For the second protocol of PBMC stimulation (1-step stimulation), PBMC (0.5×106 or 1×106) were stimulated for 1 to 7 days at 37 °C, under 5 % CO2 in a final volume of 300 μl R-10HS in 96 deep well plates (Greiner) with: (i) 2 μg/ml of LIPO-5 vaccine ; (ii) 2 μg/ml of LP mix (NeoMPS); (iii) 2 μg/ml of a pool of 36 HIV-1 15-mer peptides (NeoMPS) overlapping by 11 amino acids; (iv) 1 μg/ml of PPD (Statens Institute) and (v) 5 μg/ml of ESAT-6, a Mycobacterium tuberculosis T-specific antigen (Statens Institute). Positive and negative controls consisted of PBMC cultured with 10 ng/ml of SEB (Sigma) and PBMC cultured in R-10HS alone (non-stimulated, NS), respectively. Supernatants were harvested at days 2/11 for the re-stimulation (2-step) protocol and after various times of culture for the 1-step protocol. They were then frozen at -80 °C until assessment.
Cytokine and chemokine assays with Luminex kits
All Luminex kits used for this study are described in detail in Table 1. Luminex reproducibility was evaluated using a Luminex kit developed and shared by the Baylor Institute for Immunology Research (BIIR, Dallas, TX, USA). This kit contained polystyrene beads from BIIR and Biolegend and standards from Biolegend. We used three Milliplex MAP kits from Millipore to optimize our stimulation protocol: a 11-plex human cytokine magnetic bead panel kit (#SPR137), a 10-plex human cytokine panel kit (#SPR171), and a 10-plex human Th17 magnetic bead panel. We compared the following commercial kits: one 16-plex magnetic Procarta Immunoassays kit (Affymetrix) from Ozyme; one 15-plex Milliplex MAP human cytokine magnetic bead panel kit (#SPR158) and one simplex Milliplex MAP human cytokine/chemokine magnetic bead panel, both from Millipore; one Bio-Plex Pro assays magnetic kit and one Bio-Plex Pro human Th17 cytokine assays kit, both from Bio-Rad. A 15-plex Milliplex MAP human Th17 magnetic bead panel (Millipore, Saint-Quentin en Yvelines, France) was selected for the last part of our study; precision was evaluated using intra- and inter-variability with Millipore QCs and an internal control (pool of SEB-stimulated PBMC supernatants from the two healthy donors). All samples were acquired on a Bioplex-200 instrument (Bio-Rad, Marnes-la-Coquette, France). All experiments were performed by the same operator according to the manufacturers’ instructions.
The concentration of each analyte was obtained by interpolating FI to a dilution standard curve over at least 7 dilution points supplied with the kit and calculated using a 5PL curve by the Bio-Plex Manager 5.0 software (Bio-Rad). ICCs were used to assess reproducibility. The ICC is a measure of reliability, assessing the proportion of overall variability that is explained by the variability due to differences between samples . Generally, reliability is considered to be good if the point estimate of the intra-class correlation coefficients is at least >0.7, and ideally > 0.8. ICCs were derived from linear mixed effects models with a random effect for the intercept. The ICC was defined as the ratio of variance between supernatants to total variance. The assumption of Gaussian distributions of the residuals was verified. CVs (standard deviation over mean) were used to assess precision. A SI (calculated as a ratio between stimulated-cell supernatant concentration values and background concentration values) was used to reveal differences between the kits. All graphs were prepared using Microsoft Excel and GraphPad Prism. SAS (version 9.2, SAS Institute, Cary, North Carolina, USA) and GraphPad Prism (version 6.0, GraphPad Software, La Jolla, California, USA) were used for statistical analyses.
Five parameter log-logistic
Coefficient of variation
Enzyme-linked immunosorbent assay
Early secreted antigenic target 6 kDa protein
Human immunodeficiency virus
Intraclass correlation coefficient
Lower limit of quantification
Limit of quantification
Peripheral blood mononuclear cells
Tuberculin purified protein derivative
Staphylococcal enterotoxin B
Upper limit of quantification
We thank Corinne Krief for blood processing and PBMC storage.
We thank Sandra Zurawski from Baylor Institute for Immunology Research (BIIR, Dallas, Texas, United States of America) for providing Luminex reagents and helpful discussion.
This work was supported by the Investissements d’Avenir program managed by the ANR under reference ANR-10-LABX-77, from the Agence Nationale de Recherche contre le Sida et les hépatites (Inserm-ANRS) and from FP7-Health-2009 on African-European Research Initiative in Co-infections of Poverty Related and Neglected Infectious Diseases.
L. Richert received a PhD grant financed by Sidaction (http://www.sidaction.org/).
Availability of data and material
The datasets supporting the conclusions of this article are included within the article and its additional files.
MS and CM carried out the immunoassays and drafted the manuscript and figures. LR participated in the design of the study and performed the statistical analysis. CL, SH, RT and YL conceived the study, and participated in its design and coordination and helped to draft the manuscript. All authors read and approved the final manuscript.
The authors declare that they have no competing interests.
Consent for publication
Ethics approval and consent to participate
All patients gave written informed consent (Ethics approval: CPP Ile de France IX n° 10-023).
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Price C, Newman D. Principles and practices of immunoassays. New York: Stockton; 1997.Google Scholar
- Ellington AA, Kullo IJ, Bailey KR, Klee GG. Antibody-based protein multiplex platforms: technical and operational challenges. Clin Chem. 2010;56:186–93.View ArticlePubMedGoogle Scholar
- Fu Q, Zhu J, Van Eyk JE. Comparison of multiplex immunoassay platforms. Clin Chem. 2010;56:314–18.View ArticlePubMedGoogle Scholar
- Chowdhury F, Williams A, Johnson P. Validation and comparison of two multiplex technologies, Luminex® and Mesoscale Discovery, for human cytokine profiling. J Immunol Methods. 2009;340:55–64.View ArticlePubMedGoogle Scholar
- Crabb Breen E, Reynolds SM, Cox C, Jacobson LP, Magpantay L, Mulder CB, et al. Multisite comparison of high-sensitivity multiplex cytokine assays. Clin Vaccine Immunol. 2011;18:1229–42.View ArticleGoogle Scholar
- Prabhakar U, Eirikis E, Davis HM. Simultaneous quantification of proinflammatory cytokines in human plasma using the LabMAP™ assay. J Immunol Methods. 2002;260:207–18.View ArticlePubMedGoogle Scholar
- Ray CA, Bowsher RR, Smith WC, Devanarayan V, Willey MB, Brandt JT, et al. Development, validation, and implementation of a multiplex immunoassay for the simultaneous determination of five cytokines in human serum. J Pharm Biomed Anal. 2005;36:1037–44.View ArticlePubMedGoogle Scholar
- Dossus L, Becker S, Achaintre D, Kaaks R, Rinaldi S. Validity of multiplexed-based assays for cytokine measurements in serum and plasma from « non-diseased » subjects: Comparison with ELISA. J Immunol Methods. 2009;350:125–32.View ArticlePubMedGoogle Scholar
- Kellar KL, Kalwar RR, Dubois KA, Crouse D, Chafin WD, Kane BE. Multiplexed fluorescent bead-based immunoassays for quantification of human cytokines in serum and culture supernatant. Cytometry. 2001;45:27–36.View ArticlePubMedGoogle Scholar
- Elshal MF, McCoy JP. Multiplex bead array assays: performance evaluation and comparison of sensitivity to ELISA. Methods. 2006;38:317–23.View ArticlePubMedPubMed CentralGoogle Scholar
- Carson RT, Vignali DAA. Simultaneous quantification of 15 cytokines using a multiplexed flow cytometric assay. J Immunol Methods. 1999;227:41–52.View ArticlePubMedGoogle Scholar
- de Jager W, te Velthuis H, Prakken BJ, Kuis W, Rijkers GT. Simultaneous detection of 15 human cytokines in a single sample of stimulated peripheral blood mononuclear cells. Clin Diag Lab Immunol. 2003;10:133–9.Google Scholar
- Khan SS, Smith MS, Reda D, Suffredini AF, McCoy JP. Multiplex bead array assays for detection of soluble cytokines: comparisons of sensitivity and quantitative values among kits from multiple manufacturers. Cytometry B Clin Cytom. 2004;61B:35–9.View ArticleGoogle Scholar
- Khan IH, Krishnan VV, Ziman M, Janatpour K, Wun T, Luciw PA, et al. Comparison of multiplex suspension array large-panel kits for profiling cytokines and chemokines in rheumatoid arthritis patients. Cytometry B Clin Cytom. 2009;76B:159–68.View ArticleGoogle Scholar
- Chaturvedi AK, Kemp TJ, Pfeiffer RM, Biancotto A, Williams M, Munuo S, et al. Evaluation of multiplexed cytokine and inflammation marker measurement: a methodologic study. Cancer Epidemiol Biomarkers Prev. 2011;20:1902–11.View ArticlePubMedPubMed CentralGoogle Scholar
- Nechansky A, Grunt S, Roitt IM, Kircheis R. Comparison of the calibration standards of three commercially available multiplex kits for human cytokine measurement to WHO standards reveals striking differences. Biomark Insights. 2008;3:227–35.PubMedPubMed CentralGoogle Scholar
- Djoba Siawaya JF, Roberts T, Babb C, Black G, Golakai HJ, Stanley K, et al. An evaluation of commercial fluorescent bead-based luminex cytokine assays. PLoS One. 2008;3:e2535.View ArticlePubMedPubMed CentralGoogle Scholar
- Berthoud TK, Manaca MN, Quelhas D, Aguilar R, Guinovart C, Puyol L, et al. Comparison of commercial kits to measure cytokine responses to Plasmodium falciparum by multiplex microsphere suspension array technology. Malar J. 2011;10:115.View ArticlePubMedPubMed CentralGoogle Scholar
- Moncunill G, Aponte JJ, Nhabomba AJ, Dobaño C. Performance of multiplex commercial kits to quantify cytokine and chemokine responses in culture supernatants from Plasmodium falciparum stimulations. PLoS One. 2013;8:e52587.View ArticlePubMedPubMed CentralGoogle Scholar
- Defawe OD, Fong Y, Vasilyeva E, Pickett M, Carter DK, Gabriel E, et al. Optimization and qualification of a multiplex bead array to assess cytokine and chemokine production by vaccine-specific cells. J Immunol Methods. 2012;382:117–28.View ArticlePubMedPubMed CentralGoogle Scholar
- Salmon-Céron D, Durier C, Desaint C, Cuzin L, Surenaud M, Ben Hamouda N, et al. Immunogenicity and safety of an HIV-1 lipopeptide vaccine in healthy adults: a phase 2 placebo-controlled ANRS trial. AIDS. 2010;24:2211–23.View ArticlePubMedGoogle Scholar
- Lee JW, Weiner RS, Sailstad JM, Bowsher RR, Knuth DW, O’Brien PJ, et al. Method validation and measurement of biomarkers in nonclinical and clinical samples in drug development: a conference report. Pharm Res. 2005;22:499–511.View ArticlePubMedGoogle Scholar
- Kreher CR, Dittrich MT, Guerkov R, Boehm BO, Tary-Lehmann M. CD4+ and CD8+ cells in cryopreserved human PBMC maintain full functionality in cytokine ELISpot assays. J Immunol Methods. 2003;278:79–93.View ArticlePubMedGoogle Scholar
- Maecker HT, Hassler J, Payne JK, Summers A, Comatas K, Ghanayem M, et al. Precision and linearity targets for validation of an IFNγ ELISpot, cytokine flow cytometry, and tetramer assay using CMV peptides. BMC Immunol. 2008;9:9.View ArticlePubMedPubMed CentralGoogle Scholar
- Elsässer-Beile U, Von Kleist S, Gallati H. Evaluation of a test system for measuring cytokine production in human whole blood cell cultures. J Immunol Methods. 1991;139:191–5.View ArticlePubMedGoogle Scholar
- Desombere I, Clement F, Rigole H, Leroux-Roels G. The duration of in vitro stimulation with recall antigens determines the subset distribution of interferon-γ-producing lymphoid cells: a kinetic analysis using the interferon-γ secretion assayTM. J Immunol Methods. 2005;301:124–39.View ArticlePubMedGoogle Scholar
- Lagrelius M, Jones P, Franck K, Gaines H. Cytokine detection by multiplex technology useful for assessing antigen specific cytokine profiles and kinetics in whole blood cultured up to seven days. Cytokine. 2006;33:156–65.View ArticlePubMedGoogle Scholar
- Belabani C, Rajasekharan S, Poupon V, Johnson T, Bar-Or A, on behalf of the CIHR/MSSC NET in Clinical Autoimmunity and The Canadian Pediatric Demyelinating Disease Network. A condensed performance-validation strategy for multiplex detection kits used in studies of human clinical samples. J Immunol Methods. 2013;387:1–10.View ArticlePubMedGoogle Scholar
- Thiébaut R, Jacqmin-Gadda H. Mixed models for longitudinal left-censored repeated measures. Comput Methods Programs Biomed. 2004;74:255–60.View ArticlePubMedGoogle Scholar
- de Jager W, Bourcier K, Rijkers GT, Prakken BJ, Seyfert-Margolis V. Prerequisites for cytokine measurements in clinical trials with multiplex immunoassays. BMC Immunol. 2009;10:52.View ArticlePubMedPubMed CentralGoogle Scholar
- Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86:420–8.View ArticlePubMedGoogle Scholar