## Line Chart: Accuracy vs. Training Sequence Per Task ### Overview The image contains two line charts comparing the accuracy of different machine learning models across a sequence of tasks (T1 to T10). The charts display the performance of various continual learning algorithms, with accuracy plotted against the training sequence per task. The top chart focuses on algorithms like Finetuning, PackNet, HAT, SI, EWC, MAS, mode-IMM, LwF, and EBLL. The bottom chart focuses on Finetuning, R-PM, R-FM, GEM, and iCaRL variants. ### Components/Axes * **Y-axis (Accuracy %):** Ranges from 0 to 60, with tick marks at intervals of 10. * **X-axis (Training Sequence Per Task):** Represents the sequence of tasks from T1 to T10. Each task is separated by a vertical dashed line. * **Title (Top Chart):** Evaluation on Task * **Title (Bottom Chart):** Evaluation on Task * **Legend (Top Chart, top-right):** * `finetuning`: 21.30 (26.90) - Dotted Black Line * `joint*`: 55.70 (n/a) - Gray Line with Triangle Markers * `PackNet`: 49.13 (0.00) - Green Line with X Markers * `HAT`: 43.57 (0.00) - Pink Line with Star Markers * `SI`: 33.93 (15.77) - Orange Line * `EWC`: 42.43 (7.51) - Brown Line * `MAS`: 46.90 (1.58) - Red Line * `mode-IMM`: 36.89 (0.98) - Teal Line * `LwF`: 41.91 (3.08) - Blue Line * `EBLL`: 45.34 (1.44) - Dark Blue Line with Triangle Markers * **Legend (Bottom Chart, bottom-center):** * `finetuning`: 21.30 (26.90) - Dotted Black Line * `joint*`: 55.70 (n/a) - Gray Line with Triangle Markers * `R-PM 4.5k`: 36.09 (10.96) - Red Line with Plus Markers * `R-PM 9k`: 38.69 (7.23) - Green Dotted Line with Plus Markers * `R-FM 4.5k`: 37.31 (9.21) - Pink Dotted Line * `R-FM 9k`: 42.36 (3.94) - Gray Dotted Line * `GEM 4.5k`: 45.13 (4.96) - Light Blue Line * `GEM 9k`: 41.75 (5.18) - Dark Blue Line * `iCaRL 4.5k`: 47.27 (-1.11) - Yellow Line * `iCaRL 9k`: 48.76 (-1.76) - Red Line ### Detailed Analysis **Top Chart:** * **Finetuning (Dotted Black Line):** Starts low (around 5% at T1) and remains consistently low across all tasks, indicating poor performance. * **Joint\* (Gray Line with Triangle Markers):** Starts high (around 55% at T1) and remains relatively stable across all tasks. * **PackNet (Green Line with X Markers):** Starts high (around 50% at T1) and remains relatively stable across all tasks. * **HAT (Pink Line with Star Markers):** Starts around 50% at T1, decreases slightly, and then stabilizes. * **SI (Orange Line):** Starts around 50% at T1, decreases significantly by T2, and then continues to decrease gradually. * **EWC (Brown Line):** Starts around 40% at T1, decreases significantly by T2, and then continues to decrease gradually. * **MAS (Red Line):** Starts around 50% at T1, decreases slightly, and then stabilizes. * **mode-IMM (Teal Line):** Starts around 40% at T1, decreases slightly, and then stabilizes. * **LwF (Blue Line):** Starts around 40% at T1, decreases slightly, and then stabilizes. * **EBLL (Dark Blue Line with Triangle Markers):** Starts around 45% at T1, decreases slightly, and then stabilizes. **Bottom Chart:** * **Finetuning (Dotted Black Line):** Starts low (around 5% at T1) and remains consistently low across all tasks, indicating poor performance. * **Joint\* (Gray Line with Triangle Markers):** Starts high (around 55% at T1) and remains relatively stable across all tasks. * **R-PM 4.5k (Red Line with Plus Markers):** Starts around 50% at T1, decreases significantly by T2, and then continues to decrease gradually. * **R-PM 9k (Green Dotted Line with Plus Markers):** Starts around 40% at T1, decreases significantly by T2, and then continues to decrease gradually. * **R-FM 4.5k (Pink Dotted Line):** Starts around 45% at T1, decreases significantly by T2, and then continues to decrease gradually. * **R-FM 9k (Gray Dotted Line):** Starts around 50% at T1, decreases significantly by T2, and then continues to decrease gradually. * **GEM 4.5k (Light Blue Line):** Starts around 40% at T1, decreases slightly, and then stabilizes. * **GEM 9k (Dark Blue Line):** Starts around 40% at T1, decreases slightly, and then stabilizes. * **iCaRL 4.5k (Yellow Line):** Starts around 50% at T1, decreases slightly, and then stabilizes. * **iCaRL 9k (Red Line):** Starts around 50% at T1, decreases slightly, and then stabilizes. ### Key Observations * **Finetuning:** Consistently performs poorly across all tasks in both charts. * **Joint\*:** Maintains high accuracy across all tasks in both charts. * **Performance Drop:** Many algorithms experience a significant drop in accuracy between T1 and T2, indicating catastrophic forgetting. * **Stabilization:** After the initial drop, most algorithms stabilize, suggesting they retain some knowledge from previous tasks. * **Algorithm Comparison:** Algorithms like PackNet, HAT, MAS, EBLL, iCaRL, and GEM variants show relatively better performance compared to Finetuning, SI, EWC, and R-PM/R-FM variants. ### Interpretation The charts illustrate the challenges of continual learning, where models struggle to maintain performance on previous tasks as they learn new ones. Finetuning, a naive approach, suffers significantly from catastrophic forgetting. Joint training (Joint\*) provides a strong baseline, demonstrating the performance achievable when all data is available at once. Algorithms like PackNet, HAT, MAS, EBLL, iCaRL, and GEM variants represent more sophisticated approaches to continual learning, mitigating catastrophic forgetting to varying degrees. The initial drop in accuracy between T1 and T2 highlights the difficulty of retaining knowledge from the first task when learning subsequent tasks. The subsequent stabilization suggests that these algorithms are partially successful in preserving previously learned information.

\n ## Line Chart: Accuracy vs. Training Sequence Per Task ### Overview The image presents two line charts displaying accuracy percentages over a training sequence per task. Each chart compares the performance of several different learning algorithms (finetuning, PackNet, SI, EWC, MAS, LwF, mode-IMM, EBLL in the top chart and finetuning, R-PM 4.5k, R-PM 9k, GEM 4.5k, iCaRL 4.5k, GEM 9k, iCaRL 9k in the bottom chart). The x-axis represents the training sequence (T1 to T10), and the y-axis represents accuracy in percentage. Each line represents a different algorithm, and the charts show how accuracy changes as the training sequence progresses. Error bars are present for each data point, indicating the standard deviation. ### Components/Axes * **X-axis:** Training Sequence Per Task (T1, T2, T3, T4, T5, T6, T7, T8, T9, T10) * **Y-axis:** Accuracy (%) - Scale ranges from 0 to 60. * **Top Chart Legend (positioned at the top-center):** * finetuning: (21.30 (26.90)) - Dotted dark red line * joint*: (55.70 (n/a)) - Dotted dark blue line * PackNet: (49.13 (0.00)) - Solid green line * HAT: (43.57 (0.00)) - Solid red line * SI: (33.93 (15.77)) - Solid purple line * EWC: (42.43 (7.51)) - Solid orange line * MAS: (46.90 (1.58)) - Solid teal line * LwF: (41.91 (3.44)) - Solid light blue line * mode-IMM: (36.89 (0.98)) - Solid yellow line * EBLL: (45.34 (1.08)) - Solid pink line * **Bottom Chart Legend (positioned at the bottom-center):** * finetuning: (21.30 (26.90)) - Dotted dark red line * joint*: (55.70 (n/a)) - Dotted dark blue line * R-PM 4.5k: (36.09 (10.96)) - Solid green line * R-PM 9k: (38.69 (7.23)) - Solid red line * GEM 4.5k: (43.13 (4.96)) - Solid purple line * GEM 9k: (41.75 (5.18)) - Solid orange line * iCaRL 4.5k: (47.27 (1.11)) - Solid teal line * iCaRL 9k: (48.76 (1.76)) - Solid light blue line * **Title (positioned at the center):** "Evaluation on Task" ### Detailed Analysis or Content Details **Top Chart:** * **finetuning (dark red, dotted):** Starts around 20% at T1, fluctuates between 20-30% throughout the training sequence, with a slight upward trend towards T10. * **joint* (dark blue, dotted):** Starts around 50% at T1, remains relatively stable around 50-60% throughout the training sequence. * **PackNet (green, solid):** Starts around 40% at T1, increases to approximately 50% by T4, then fluctuates between 45-55% for the remainder of the sequence. * **HAT (red, solid):** Starts around 35% at T1, increases to approximately 45% by T3, then fluctuates between 40-50% for the remainder of the sequence. * **SI (purple, solid):** Starts around 25% at T1, increases to approximately 35% by T3, then fluctuates between 30-40% for the remainder of the sequence. * **EWC (orange, solid):** Starts around 35% at T1, increases to approximately 45% by T4, then fluctuates between 40-50% for the remainder of the sequence. * **MAS (teal, solid):** Starts around 40% at T1, increases to approximately 50% by T4, then fluctuates between 45-55% for the remainder of the sequence. * **LwF (light blue, solid):** Starts around 35% at T1, increases to approximately 45% by T4, then fluctuates between 40-50% for the remainder of the sequence. * **mode-IMM (yellow, solid):** Starts around 30% at T1, increases to approximately 40% by T4, then fluctuates between 35-45% for the remainder of the sequence. * **EBLL (pink, solid):** Starts around 40% at T1, increases to approximately 50% by T4, then fluctuates between 45-55% for the remainder of the sequence. **Bottom Chart:** * **finetuning (dark red, dotted):** Similar to the top chart, starts around 20% at T1, fluctuates between 20-30% throughout the training sequence. * **joint* (dark blue, dotted):** Similar to the top chart, starts around 50% at T1, remains relatively stable around 50-60% throughout the training sequence. * **R-PM 4.5k (green, solid):** Starts around 30% at T1, increases to approximately 40% by T4, then fluctuates between 35-45% for the remainder of the sequence. * **R-PM 9k (red, solid):** Starts around 30% at T1, increases to approximately 40% by T4, then fluctuates between 35-45% for the remainder of the sequence. * **GEM 4.5k (purple, solid):** Starts around 35% at T1, increases to approximately 45% by T4, then fluctuates between 40-50% for the remainder of the sequence. * **GEM 9k (orange, solid):** Starts around 35% at T1, increases to approximately 45% by T4, then fluctuates between 40-50% for the remainder of the sequence. * **iCaRL 4.5k (teal, solid):** Starts around 40% at T1, increases to approximately 50% by T4, then fluctuates between 45-55% for the remainder of the sequence. * **iCaRL 9k (light blue, solid):** Starts around 40% at T1, increases to approximately 50% by T4, then fluctuates between 45-55% for the remainder of the sequence. ### Key Observations * The "joint*" method consistently achieves the highest accuracy across both charts, remaining stable around 55-60%. * "finetuning" consistently shows the lowest accuracy, fluctuating between 20-30%. * The performance of most algorithms tends to plateau after T4, with fluctuations around a certain accuracy level. * The error bars indicate variability in performance, but the general trends remain consistent. * The 9k versions of R-PM, GEM, and iCaRL generally perform slightly better than their 4.5k counterparts. ### Interpretation The charts demonstrate the performance of various continual learning algorithms on a task. The "joint*" method appears to be the most effective, maintaining high accuracy throughout the training sequence. This suggests that the joint training approach is well-suited for this particular task. "finetuning" consistently underperforms, indicating that it struggles to retain knowledge from previous tasks as new tasks are introduced. The other algorithms show intermediate performance, with varying degrees of success. The slight improvement observed with the 9k versions suggests that increasing the model capacity can lead to better performance, but the gains are not substantial. The error bars highlight the inherent variability in machine learning performance, and it is important to consider these uncertainties when interpreting the results. The plateauing of accuracy after T4 suggests that the algorithms may be reaching their learning capacity or that the task becomes saturated. Overall, the data provides valuable insights into the strengths and weaknesses of different continual learning algorithms and can guide the selection of appropriate methods for specific applications.

\n ## Line Charts: Continual Learning Method Accuracy Across Sequential Tasks ### Overview The image contains two vertically stacked line charts comparing the performance of various continual learning algorithms. Both charts plot "Accuracy %" against a "Training Sequence Per Task" (T1 through T10). Each chart has its own legend listing different methods and their associated performance metrics. The charts appear to be from a technical paper evaluating how well different machine learning models retain knowledge when learning new tasks sequentially. ### Components/Axes **Common Elements (Both Charts):** * **X-Axis:** Labeled "Training Sequence Per Task". Major tick marks are labeled T1, T2, T3, T4, T5, T6, T7, T8, T9, T10. * **Y-Axis:** Labeled "Accuracy %". Scale runs from 0 to 60, with major ticks at 0, 10, 20, 30, 40, 50, 60. * **Plot Title:** "Evaluation on Task" is centered above each plot area. * **Grid:** Light gray vertical and horizontal grid lines are present. **Top Chart Legend (Positioned at the top of the image, spanning horizontally):** The legend lists 10 methods. Each entry shows a line style/marker, the method name, a primary number (likely mean accuracy), and a number in parentheses (likely standard deviation or another metric). 1. `--- finetuning: 21.30 (26.90)` (Black, dotted line) 2. `--- PackNet: 49.13 (0.00)` (Green line with 'x' markers) 3. `--- SI: 33.93 (15.77)` (Orange line) 4. `--- MAS: 46.90 (1.58)` (Red line) 5. `--- LwF: 41.91 (3.08)` (Light blue line) 6. `--- HAT: 43.57 (0.00)` (Pink line with '+' markers) 7. `--- EWC: 42.43 (7.51)` (Yellow line) 8. `--- mode-IMM: 36.89 (0.98)` (Brown line) 9. `--- EBLL: 45.34 (1.44)` (Dark blue line) 10. `--- joint*: 55.70 (n/a)` (Gray line with right-pointing triangle markers) **Bottom Chart Legend (Positioned below the top chart's plot area):** The legend lists 10 methods, comparing variants with "4.5k" and "9k" (likely memory buffer sizes). 1. `--- finetuning: 21.30 (26.90)` (Black, dotted line) - *Same as top chart.* 2. `--- R-PM 4.5k: 36.09 (10.96)` (Light green, dashed line) 3. `--- R-FM 4.5k: 37.31 (9.21)` (Pink, dashed line) 4. `--- GEM 4.5k: 45.13 (4.96)` (Light blue line) 5. `--- iCaRL 4.5k: 47.27 (-1.11)` (Orange line) 6. `--- R-PM 9k: 38.69 (7.23)` (Green, dashed line with 'x' markers) 7. `--- R-FM 9k: 42.36 (3.94)` (Pink, dashed line with '+' markers) 8. `--- GEM 9k: 41.75 (5.18)` (Dark blue line) 9. `--- iCaRL 9k: 48.76 (-1.76)` (Red line) 10. `--- joint*: 55.70 (n/a)` (Gray line with right-pointing triangle markers) - *Same as top chart.* ### Detailed Analysis **Top Chart - Method Comparison:** * **Trend Verification & Data Points:** * **finetuning (Black dotted):** Starts near 50% at T1, then plummets dramatically to below 10% by T2 and remains very low (~5-15%) for all subsequent tasks. This shows catastrophic forgetting. * **PackNet (Green, 'x'):** Maintains a very stable, high accuracy (~50%) across all tasks T1-T10. The line is nearly flat. * **MAS (Red):** Starts high (~50%), shows a slight, gradual decline but remains above 40% through T10. * **EBLL (Dark blue):** Similar trend to MAS, starting high and showing a moderate decline, ending in the mid-40s%. * **HAT (Pink, '+'):** Starts high, shows a slight decline, and appears to stabilize around 40-45%. * **LwF (Light blue):** Starts high, declines more noticeably than MAS/EBLL, ending near 40%. * **EWC (Yellow):** Starts high, shows a significant decline, dropping to around 30% by T5 before a slight recovery. * **SI (Orange):** Shows a steep decline similar to EWC, dropping from ~50% to the 20-30% range. * **mode-IMM (Brown):** Starts lower than others (~45%), declines steadily to the 30-40% range. * **joint* (Gray, triangles):** This appears to be an upper-bound baseline. It maintains a very high accuracy (~55%) across all tasks, consistently above all other methods. **Bottom Chart - Memory Size Comparison (4.5k vs. 9k):** * **Trend Verification & Data Points:** * **finetuning (Black dotted):** Identical catastrophic forgetting trend as in the top chart. * **joint* (Gray, triangles):** Identical high-performance baseline as in the top chart (~55%). * **iCaRL (Orange 4.5k / Red 9k):** Both variants perform well. The 9k version (Red) consistently outperforms the 4.5k version (Orange) by a small margin (approx. 1-3%), maintaining accuracy near 50%. * **GEM (Light blue 4.5k / Dark blue 9k):** The 4.5k version (Light blue) appears to have a slight edge over the 9k version (Dark blue) in later tasks, which is counter-intuitive. Both fluctuate between 35-50%. * **R-FM (Pink dashed 4.5k / Pink dashed '+' 9k):** The 9k version shows a clear improvement over the 4.5k version, with the 9k line staying above 40% and the 4.5k line dropping into the 30s%. * **R-PM (Light green dashed 4.5k / Green dashed 'x' 9k):** The 9k version shows a notable improvement over the 4.5k version. The 4.5k line drops significantly after T1, while the 9k line maintains higher accuracy. ### Key Observations 1. **Catastrophic Forgetting:** The `finetuning` method serves as a baseline for catastrophic forgetting, showing a severe and permanent drop in accuracy after the first task. 2. **Upper Bound:** The `joint*` method (likely trained on all data simultaneously) represents a performance ceiling (~55%) that sequential methods approach but do not exceed. 3. **Top Performers:** In the top chart, `PackNet` is the most stable and highest-performing sequential method, followed by `MAS` and `EBLL`. In the bottom chart, `iCaRL 9k` is the top-performing sequential method. 4. **Memory Benefit:** For `R-PM` and `R-FM`, increasing the memory buffer from 4.5k to 9k provides a clear performance benefit. The effect is less consistent or even reversed for `GEM` in this specific evaluation. 5. **Stability vs. Decline:** Methods like `PackNet` show remarkable stability (flat lines), while others like `SI` and `EWC` show a pronounced downward trend as more tasks are learned. ### Interpretation This data demonstrates the core challenge of continual learning: preventing performance degradation on previously learned tasks while acquiring new ones. The stark contrast between `finetuning` and all other methods highlights the necessity of specialized algorithms. The charts suggest that **regularization-based methods** (like MAS, EWC, SI) offer a middle ground, mitigating forgetting but often still experiencing a gradual decline. **Replay-based methods** (like iCaRL, GEM, R-PM, R-FM) show that maintaining a memory buffer is effective, and its size (4.5k vs. 9k) can be a critical hyperparameter, though its impact varies by algorithm. **Architectural methods** (like PackNet, which likely allocates separate network parameters per task) can achieve near-perfect knowledge retention, as shown by its flat line, but may have other trade-offs not measured here (e.g., model size). The `joint*` baseline is crucial for context; it shows the maximum achievable accuracy if the model had access to all data at once. The gap between the best sequential methods (e.g., PackNet, iCaRL 9k) and this baseline represents the "cost" of learning sequentially. The goal of continual learning research, as visualized here, is to close this gap. The variability in performance across methods and tasks (seen in the jagged lines and different slopes) indicates that no single method is universally optimal, and performance is highly dependent on the specific algorithm and its configuration.

## Line Graph: Accuracy Trends Across Continual Learning Tasks ### Overview The image displays two line graphs comparing the accuracy of various continual learning methods across 10 tasks (T1–T10). Each graph tracks accuracy (%) over training sequences per task, with distinct data series represented by colored lines. The top graph includes methods like PackNet, SI, MAS, and EWC, while the bottom graph focuses on R-PM, R-FM, GEM, and iCaRL variants. A "joint*" baseline (dashed black line) is present in both graphs. --- ### Components/Axes - **X-axis**: "Training Sequence Per Task" (T1–T10), marked with vertical dashed lines. - **Y-axis**: "Accuracy %" (0–60%). - **Legends**: - **Top Legend** (left-aligned): - finetuning (black dotted) - PackNet (green) - SI (orange) - MAS (red) - EWC (yellow) - mode-IMM (brown) - LwF (blue) - EBLL (purple) - joint* (dashed black) - **Bottom Legend** (left-aligned): - finetuning (black dotted) - R-PM 4.5k (green dotted) - R-FM 4.5k (pink dotted) - R-PM 9k (green dash-dot) - R-FM 9k (pink dash-dot) - GEM 4.5k (blue) - GEM 9k (purple) - iCaRL 4.5k (orange) - iCaRL 9k (red) - joint* (dashed black) --- ### Detailed Analysis #### Top Graph Trends 1. **finetuning (black dotted)**: Starts at ~50% (T1) but drops sharply to ~10% by T10, showing catastrophic forgetting. 2. **PackNet (green)**: Maintains ~45–49% accuracy across all tasks, with minimal fluctuation. 3. **SI (orange)**: Peaks at ~55% (T1) but declines to ~30% by T10, with erratic drops. 4. **MAS (red)**: Stable at ~45–47% until T5, then declines to ~35% by T10. 5. **EWC (yellow)**: Declines from ~42% (T1) to ~25% (T10), with gradual drops. 6. **mode-IMM (brown)**: Starts at ~36% (T1), rises to ~40% (T3), then falls to ~25% (T10). 7. **LwF (blue)**: Stable at ~40–42% until T5, then declines to ~30% (T10). 8. **EBLL (purple)**: Peaks at ~45% (T1), drops to ~35% (T5), and stabilizes at ~30% (T10). 9. **joint* (dashed black)**: Consistently ~55% accuracy across all tasks. #### Bottom Graph Trends 1. **finetuning (black dotted)**: Same as top graph (~50% → ~10%). 2. **R-PM 4.5k (green dotted)**: Starts at ~36% (T1), drops to ~25% (T10). 3. **R-FM 4.5k (pink dotted)**: Peaks at ~37% (T1), declines to ~28% (T10). 4. **R-PM 9k (green dash-dot)**: Starts at ~38% (T1), drops to ~28% (T10). 5. **R-FM 9k (pink dash-dot)**: Peaks at ~42% (T1), declines to ~30% (T10). 6. **GEM 4.5k (blue)**: Starts at ~45% (T1), drops to ~35% (T10). 7. **GEM 9k (purple)**: Starts at ~41% (T1), declines to ~30% (T10). 8. **iCaRL 4.5k (orange)**: Peaks at ~47% (T1), declines to ~35% (T10). 9. **iCaRL 9k (red)**: Peaks at ~48% (T1), declines to ~35% (T10). 10. **joint* (dashed black)**: Consistent ~55% accuracy. --- ### Key Observations 1. **joint* Baseline**: Dominates both graphs, maintaining ~55% accuracy, suggesting it represents a robust hybrid approach. 2. **Catastrophic Forgetting**: Methods like finetuning and SI show severe accuracy drops, indicating poor retention of prior tasks. 3. **Stability**: PackNet (top) and iCaRL 9k (bottom) exhibit the most stability, retaining >30% accuracy across tasks. 4. **Performance Gaps**: Top graph methods (e.g., PackNet) outperform bottom graph methods (e.g., R-PM), suggesting architectural differences. 5. **Task-Specific Declines**: Most methods degrade after T5, with sharper drops in later tasks (T7–T10). --- ### Interpretation The data highlights the challenges of continual learning, where methods must balance new task learning with prior knowledge retention. The **joint*** baseline’s consistent performance implies it effectively mitigates catastrophic forgetting, possibly through task-agnostic or modular design. - **Top Graph**: Methods like PackNet and MAS prioritize stability, while SI and EWC struggle with task shifts. The absence of R-PM/R-FM variants here suggests a focus on different architectures (e.g., memory vs. regularization). - **Bottom Graph**: R-PM/R-FM and GEM/iCaRL variants show moderate performance, with iCaRL 9k outperforming others. The inclusion of "4.5k" and "9k" likely refers to parameter counts (e.g., 4.5k vs. 9k parameters), with larger models (9k) performing slightly better but still declining over tasks. **Notable Anomalies**: - SI’s erratic drops (e.g., T3–T4) suggest sensitivity to hyperparameters or task order. - EBLL’s sharp decline after T5 may indicate overfitting to early tasks. This analysis underscores the need for methods that dynamically adapt to task complexity and preserve knowledge across diverse scenarios.