## Line Chart: Cost per Sequence vs. Number of Items per Sequence
### Overview
The chart compares the cost per sequence (in bits) for three different models as the number of items per sequence increases from 6 to 20. The models are:
1. **LSTM** (blue line)
2. **NTM with LSTM Controller** (green line)
3. **NTM with Feedforward Controller** (red line)
### Components/Axes
- **X-axis**: "number of items per sequence" (ranges from 6 to 20, with markers at 6, 8, 10, 12, 14, 16, 18, 20).
- **Y-axis**: "cost per sequence (bits)" (ranges from 0 to 40, with markers at 0, 5, 10, 15, 20, 25, 30, 35, 40).
- **Legend**: Located on the right side of the chart, with color-coded labels:
- **Blue**: LSTM
- **Green**: NTM with LSTM Controller
- **Red**: NTM with Feedforward Controller
### Detailed Analysis
1. **LSTM (Blue Line)**:
- Starts at ~2 bits when the number of items is 6.
- Increases sharply to ~35 bits at 10 items.
- Plateaus slightly above 35 bits for sequences with 12–20 items.
- **Key Trend**: Steep initial rise followed by stabilization.
2. **NTM with LSTM Controller (Green Line)**:
- Starts at ~0 bits for 6 items.
- Gradually increases to ~5 bits at 20 items.
- **Key Trend**: Slow, linear growth.
3. **NTM with Feedforward Controller (Red Line)**:
- Starts at ~0 bits for 6 items.
- Increases to ~7 bits at 20 items.
- **Key Trend**: Slightly steeper than the green line but remains below the blue line.
### Key Observations
- The **LSTM** model exhibits a significantly higher cost per sequence compared to the NTM variants, especially for sequences with 10 or more items.
- The **NTM with LSTM Controller** and **NTM with Feedforward Controller** show similar trends but differ slightly in cost, with the Feedforward Controller being marginally more expensive.
- All models show minimal cost increases for sequences with fewer than 10 items, but divergence occurs beyond this threshold.
### Interpretation
The data suggests that **LSTM** incurs a high computational cost early in the sequence but stabilizes, making it less efficient for longer sequences. In contrast, **NTM variants** (with LSTM or Feedforward Controllers) demonstrate lower and more scalable costs, indicating better performance for longer sequences. The NTM with LSTM Controller is the most cost-effective, while the Feedforward Controller variant is slightly less efficient but still outperforms the standard LSTM. This implies that NTM architectures with specialized controllers may offer a better trade-off between complexity and cost for sequence modeling tasks.
