## Line Chart: Latency per batch vs. Throughput
### Overview
The image is a line chart comparing the latency per batch (in milliseconds) against throughput (in samples per second) for two different methods: AR and SpecDec. The chart shows how latency changes as throughput increases for each method.
### Components/Axes
* **X-axis:** Throughput (samples/s), ranging from 0 to 250.
* **Y-axis:** Latency per batch (ms), ranging from 0 to 700.
* **Legend:** Located in the top-right corner.
* Blue line: AR
* Orange line: SpecDec
* **Gridlines:** Present in the background.
### Detailed Analysis
* **AR (Blue Line):** The latency increases significantly with throughput.
* At approximately 15 samples/s, latency is around 400 ms, batch size 8.
* At approximately 30 samples/s, latency is around 470 ms, batch size 16.
* At approximately 80 samples/s, latency is around 570 ms, batch size 32.
* At approximately 90 samples/s, latency is around 710 ms, batch size 64.
* At approximately 10 samples/s, latency is around 330 ms, batch size 4.
* At approximately 5 samples/s, latency is around 230 ms, batch size 1.
* **SpecDec (Orange Line):** The latency increases slightly with throughput.
* At approximately 240 samples/s, latency is around 130 ms, batch size 32.
* At approximately 150 samples/s, latency is around 100 ms, batch size 16.
* At approximately 80 samples/s, latency is around 90 ms, batch size 8.
* At approximately 50 samples/s, latency is around 70 ms, batch size 4.
* At approximately 10 samples/s, latency is around 50 ms, batch size 1.
### Key Observations
* AR exhibits a much higher latency than SpecDec across all throughput values.
* The latency of AR increases sharply as throughput increases.
* The latency of SpecDec remains relatively stable as throughput increases.
* Batch sizes are annotated next to each data point on both lines.
### Interpretation
The chart demonstrates that SpecDec is significantly more efficient than AR in terms of latency for a given throughput. The AR method's latency is highly sensitive to changes in throughput, whereas SpecDec maintains a relatively low and stable latency even as throughput increases. This suggests that SpecDec is a more scalable solution for applications where high throughput is required. The batch sizes are annotated next to each data point, indicating the batch size used for each throughput value.
