Can multimodal AI read values and trends from a graph accurately?

Can multimodal AI read values and trends from a graph accurately? · Student Data Sheet

Question: For a fixed set of verified graphs, how accurately does one multimodal AI model report plotted values, trend direction and change?

Hypothesis: If the chart contains more visual features such as a legend or a nonzero axis start, then the model's value-reading and trend answers may differ from simpler charts, because a chart answer depends on reading axes and visual encodings before doing the requested comparison, but this test does not predict a result.

Materials

• teacher-approved AI tool that accepts chart images
• 12 teacher-made line charts from saved source tables
• printed or digital answer key checked by two people
• same computer and fixed image-resolution export
• 36-row raw-answer sheet
• teacher approval for the tool and its privacy settings

Safety before starting

• Use public, synthetic, or teacher-created charts. Remove names, health information, grades, account data, and confidential details before sharing a chart with an AI service.
• Verify every reported value and trend against the original axes, units, legend, and source key. Do not rely on model output for consequential medical, legal, financial, or safety decisions.

Lab notes

• Do not use a model's graph reading as the sole basis for lab conclusions, grades, medical choices, financial decisions or safety actions.

Variables

Independent: Chart feature group · Dependent: Agreement with the chart's verified source data

Keep constant: same model/version and settings · same fixed prompt · web search and tools off · 12 teacher-checked charts · same three question types per chart · answer key frozen first · consistent image resolution

Procedure

1. Build a verified chart set: With a teacher, create 12 line charts from small, invented classroom datasets whose exact values are saved in a source table. Make four simple single-series charts, four two-series charts with clear legends, and four single-series charts with nonzero y-axis starts. Keep chart titles, fonts and image size readable. Have two people check every plotted point, axis tick, unit and legend against the source table before testing.
2. Write questions and freeze the key: For each chart, prepare the same three question types: read one specified point, state whether the selected series rises/falls/stays level from the first to last point, and calculate the endpoint change. Record exact answers and units from the source table before any AI run. Avoid questions where a point falls between ticks unless a clear tolerance is defined.
3. Fix the model and prompt: Use one teacher-approved image-capable AI tool. Record its name, visible model/version, date, image resolution and settings; turn off browsing, code execution and external tools if the product allows. Use one fixed prompt requesting a concise value with unit or one of the trend labels. If a setting cannot be disabled or verified, note that limit.
4. Test all charts in mixed order: Use a fresh chat for each chart and ask its three prewritten questions without supplying the source table. Randomize the 12 chart IDs in advance. Do not correct the model or reveal answers between charts. Save the complete answer, including refusals, missing units and uncertainty; do not count an unsupported guess as correct.
5. Score numbers and trends separately: Compare every number with the frozen source key. Record absolute error in the chart's units; before testing, set a tolerance no wider than half the smallest labeled grid interval. Score direction/comparison as correct or incorrect. For two-series charts, check that the named series matches the legend. Keep unreadable or ambiguous answers in the raw record and mark them separately.
6. Compare chart groups carefully: Each of the three chart groups contains four charts and 12 questions. Show counts and the raw error values before percentages or averages. Compare question types separately, then describe the small group patterns. The charts were made for this class, so repeated templates and four charts per group cannot establish a general chart-reading ability.

Record your observations

Summarize the three chart feature groups

Check each graph before asking the model

Formula: Numeric absolute error = |AI-reported value − source-table value| (in the graph's units)

Answer key and expected pattern

A model can answer a trend question correctly while misreading an exact value; score them separately.

A legend or nonzero axis may change what must be interpreted before answering; this test does not assume which group will be harder.

Four charts per group are descriptive and cannot establish overall multimodal AI accuracy.

Remember: Trace the plotted point back to the axis and source table. A plausible graph answer is only reliable when the value, units, series and scale all match.