The short answer
Meta-analysis tools do different jobs: AI can propose extracted values or code, while statistical software calculates a specified model. Choose an integrated workflow or a script-based setup according to the methods and records you need to inspect. This guide compares their roles and follows three fictional studies through a data ledger, an effect-direction check, and an analysis handoff.
Key takeaways
Compare extraction assistance, integrated analysis, and statistical packages by the output you need to verify.
Keep review eligibility separate from readiness for a particular analysis: a wrong time point or missing SD can prevent pooling without removing the study from the review.
Save checked inputs, explicit settings, the execution record, and outputs so another analyst can assess or rerun the calculation.
Gatsbi publishes this guide and makes some of the tools discussed. We use the same task-based criteria for Gatsbi and other products, drawing on their public documentation. These are not hands-on benchmark results; check current features and access before choosing.
Choose an analysis route before choosing a product
First specify the question, study designs, outcome, comparison, and time point. Then identify the effect measure and the data its calculation requires. For the concepts behind these choices, read what meta-analysis does. Software selection follows the intended analysis.
An integrated workflow can connect extraction, analysis, plots, and drafting without writing a script. A script-based route exposes code and package arguments for inspection and reuse. Either route must support your actual design; dependent effects, cluster trials, and multi-arm studies require more than a generic meta-analysis label.
Separate the four stages below when deciding what to delegate. A correct extracted number does not establish that a transformation or model is appropriate. If selection and source extraction are still unresolved, complete that work in the systematic-review tools guide.
| Stage | Possible assistance | Acceptance evidence |
|---|---|---|
Read reports | Propose numbers and source passages | Checked study, arm, outcome, statistic, unit, and time point |
Prepare effects | Draft a transformation or calculation | Documented method and an independently checked example |
Fit models | Execute a statistical package or integrated analysis | Explicit settings, suitable design handling, and inspectable results |
Interpret | Draft an explanation of the output | Statements consistent with the estimate, uncertainty, and study limitations |
Compare inputs, analysis controls, and reproducibility
The named examples below are described from provider documentation checked on 18 October 2026. Gatsbi publishes this guide and is included under the same criteria. These are documented roles, not results from a comparative accuracy test. The coding-assistant row is a category to evaluate, not a reviewed product.
RevMan and metafor are review-analysis or statistical software; using them alongside an assistant does not make them generative AI. Conversely, an extraction assistant does not become a complete analysis engine because it produces a table. Decide which component will calculate your effect measures and models.
For each candidate, list the required effect measures, study designs, estimators, and interval methods. Confirm them in the relevant version's documentation, then inspect a small output. Where the overview does not establish a feature or export detail, keep it marked for verification rather than assuming support or absence.
| Role / source | Input and documented output | Useful when | What your trial must establish |
|---|---|---|---|
Extraction assistance / Elicit | Papers in its structured review workflow can be screened and used for data extraction. | You need proposed fields before preparing the analysis dataset. | Source locations, arm/time-point mapping, and an export that can be checked; extraction alone does not establish valid analysis inputs. |
Review-analysis environment / RevMan | Study data feed review management, analysis, and reporting. | You want analysis within a review environment. | Support for the intended designs and methods, visible settings, and the data/output records needed to reproduce your result. |
Statistical package / R with metafor | Numerical data and explicit function arguments produce effect calculations, fitted models, and plots. | Your team can inspect and maintain an R analysis script. | Correct function and argument choices, dependence handling, package versions, and an independent rerun. |
Integrated AI-assisted workflow / Gatsbi Reviewer · publisher | Product descriptions include quantitative extraction, fixed-effect and random-effects analysis, forest/funnel plots, I²/Q, and raw-data/key-result export. | You want to connect reviewed study data with analysis and a manuscript draft. | Required effect measures and complex designs, exposed estimators/interval settings, source mapping, and whether exports permit independent checking. The overview does not establish all of these. |
Coding-assistant category | Prompts and supplied data may be used to propose editable code; behavior depends on the chosen product and configuration. | An analyst can review the proposed script against the intended method. | The exact model, input mapping, environment, and output. No specific coding assistant is benchmarked here. |
Build a ledger for the intended analysis
Use separate identifiers for the study, its reports, and the analysis. Preserve original values alongside checked values and transformations. A study may be eligible for the review but lack the data needed for one outcome or follow-up period; record those as separate decisions.
The fictional ledger below uses analysis ID MD_12W: intervention minus comparator mean difference in a continuous score at 12 weeks, with lower scores indicating better outcomes. Study A is the same teaching report used in the screening and extraction guide. Studies B and C introduce a different time point and a missing standard deviation. These are invented inputs, not observed study findings or product outputs.
Study A supplies the displayed fields for the planned calculation. Study B's six-week means cannot silently become 12-week values. Study C requires a documented missing-data decision: seek the value, use a justified derivation or planned analysis approach where appropriate, or omit it from this quantitative synthesis. None of those choices automatically removes an otherwise eligible study from the review.
Identity and scope: study_id, report_id, analysis_id, arm, outcome, timepoint_weeks, statistic, and unit.
Values and provenance: original_value, checked_value, sample size (n), source_location, transformation, and missing_reason. Display a missing SD as not reported, not zero.
Decision: verification_status and include_in_analysis, with the reason and reviewer. Keep include_in_review separate so a data problem cannot silently become a study exclusion.
| Study / source | Reported information | Check or correction | Readiness for MD_12W |
|---|---|---|---|
A / Report A, table 2, week 12 | Intervention: n=40, mean=12, SD=4; comparator: n=40, mean=15, SD=5 | Confirm arm labels, statistic, and units against the source; retain any extraction correction. | Displayed fields are complete for the planned calculation; verify the study design and analysis assumptions before inclusion. |
B / Report B, table 3, week 6 | Intervention mean=10; comparator mean=12 | Follow-up differs from the 12-week definition; retain the actual time point. | Not ready for this analysis. Resolve timing and any other required fields; do not silently pool. |
C / Report C, table 2, week 12 | Intervention mean=11; comparator mean=14; one required SD is not reported | Keep the missing SD visible and record the chosen response. | Not ready. Do not invent an SD or interpret a blank as zero. |
Verify code or integrated analysis settings
Begin with the mapping from fields to the calculation. Are these final scores or changes from baseline? Are intervention and comparator in the intended order? Are SDs distinguished from standard errors and confidence intervals? Cochrane's effect-measure guidance explains the methods context; the software's documentation governs its implementation.
Recalculate fictional Study A independently: intervention minus comparator at 12 weeks is 12 − 15 = −3 points. Reversing the groups gives +3 from the same numbers. Check that the script or interface preserves the planned order before interpreting the direction of the result.
Same numbers, opposite effect direction
In fictional Study A at 12 weeks, the intervention mean is 12 and the comparator mean is 15. The planned intervention-minus-comparator difference is minus 3 points. Reversing the groups incorrectly gives plus 3. A number line places these results on opposite sides of zero; uncertainty and model choices still require verification.
Teaching example · fictional Study A
Study A · table 2 · week 12
Intervention − comparator
Comparator − intervention
Mean difference (points) · 0 = no difference in means
The source values stay the same. Swapping the groups flips the sign.
Verify arm labels and the planned subtraction order. A correct −3 does not verify standard deviations, sample sizes, uncertainty or the analysis model.
For code, inspect the function arguments, data mapping, missing-value handling, package dependencies, and runtime versions. For an integrated interface, retain the entered data, selected settings, exported result, and available version information. If the estimator, interval method, or other necessary setting is not exposed, record the limitation and seek clarification.
Then check the uncertainty calculation and model for the study design. Paired, clustered, repeated, and multi-arm data may need different treatment. The arithmetic illustration checks one subtraction only; this guide does not select a pooling model or validate an uncertainty calculation for the fictional studies.
| Setting | Recorded choice or status | Acceptance check |
|---|---|---|
Analysis and outcome | MD_12W; continuous score at 12 weeks | Matches the question and ledger |
Direction | Intervention minus comparator | Study A gives −3, not +3 |
Statistic and units | Group means; points; SDs and sample sizes retained | Final scores, SDs, and units have not been confused with another quantity |
Model, estimator, and interval method | Not selected or calculated in this example | Specify and verify these for the real project; do not inherit unexamined defaults |
Software and execution | Record actual product/package, version, settings, and run date | Another analyst can identify what was run and what remains unavailable |
Reconcile the output before writing the result
Compare the output with the checked dataset and the intended analysis. Reconcile study and participant counts, effect direction, intervals, weights, and labels. Investigate a surprising value before drafting the results paragraph.
In this ledger, only A supplies the displayed fields for the specified calculation. B has the wrong follow-up and C lacks a required SD, so the inputs do not justify a three-study pooled estimate. A program that nevertheless produces one requires investigation of its inclusion rules and missing-data behavior.
Interpretation still needs the study context: an I² threshold alone should not choose the model, and a nonsignificant result does not prove no effect. The meta-analysis explainer covers those concepts, with further methods in Cochrane's analysis chapter. If pooling cannot answer the question, choose the appropriate output and handoff in the evidence-synthesis guide.
Every contributing row belongs to the intended outcome, comparison, and follow-up.
Multiple reports or effects from the same participants have not been treated as independent by accident.
Transformations, missing-data decisions, and sensitivity analyses are visible beside the affected results.
Manuscript values and labels match the saved output; conclusions retain uncertainty and study limitations.
Hand over inputs, settings, and results together
A colleague should be able to follow the analysis from permitted source records to reported results. The manifest below is a structure you can copy into your own project, not a claim that every candidate exports each item. Keep raw exports, checked data, and analysis inputs separate.
For a proposed Gatsbi trial, start with checked study fields and a specified analysis. Review and edit the output, then export the raw data and key results described on the product page. Compare those exports with this manifest. Source mapping, full model settings, code access, and a frozen environment remain items to verify; a report or figure alone does not establish reproducibility.
Where code is available, have another analyst rerun it from the saved inputs. For an interface-based workflow, have them reconstruct the settings and independently check key calculations where feasible. Record any part that cannot be reproduced, including an unavailable service version. Save corrections as new versions with reasons, and reconcile the final output with the manuscript.
| Record / suggested name | Must contain | Handoff check |
|---|---|---|
README.txt | Question, analysis IDs, execution entry point or interface steps, prerequisites, and limitations | A colleague knows how to start and what access is required |
source-manifest.csv and extraction-ledger.csv | Study/report IDs, permitted source locations, original/checked values, and correction reasons | Each input can be traced without redistributing restricted full texts |
analysis-input.csv and data dictionary | Only the intended analysis inputs, units, missingness, and transformations | No unexplained inclusion, invented value, or changed time point |
analysis-config.json or equivalent record | Effect measure, direction, model, estimator, interval method, and missing-data rules | The method is explicit; unavailable settings are identified |
analysis.R or run-record.md; environment.txt | Checked code or recorded interface steps, software/package versions, and run date | Another analyst can rerun or reconstruct the calculation and identify remaining limits |
decision-log.csv and outputs | Reasons for changes, sensitivity runs, saved tables/figures, and manuscript checks | Reported values match the approved output version |
Common questions
Are RevMan and metafor AI tools?
They are review-analysis or statistical software. They can be used alongside AI extraction or coding assistance, but those roles should be evaluated separately.
When should I choose an integrated workflow instead of R?
Consider an integrated workflow when its supported methods and exposed records meet your analysis needs and you prefer an interface. Consider a script-based setup when your team needs and can maintain inspectable code or particular methods. Verify the required design, settings, and exports for either route.
What should I check in AI-generated analysis code?
Check the variables, group direction, effect measure, uncertainty inputs, dependence among effects, missing-data rules, model arguments, and package versions. Recalculate a simple example independently and have another analyst rerun the saved script.
Can a study stay in the review if it cannot enter the meta-analysis?
Yes. Review eligibility and contribution to a particular analysis are separate decisions. A missing SD or incompatible follow-up may prevent one calculation while the study remains relevant to the review; record the reason and use an appropriate synthesis approach.
Sources & further reading
Follow the original sources for their complete methods, scope, and latest updates.
- Choosing effect measuresCochrane
- Analysing data and undertaking meta-analysesCochrane
- Choosing an Elicit workflowElicit
- RevManCochrane
- metafor documentationmetafor project
- Gatsbi Reviewer analysis and export workflowGatsbi · publisher product documentation