## Line Chart: Evaluation on Task ### Overview The image is a line chart comparing the performance of different continual learning algorithms across a sequence of tasks (T1 to T8). The y-axis represents accuracy percentage, and the x-axis represents the training sequence per task. Each line represents a different algorithm, and the chart shows how the accuracy of each algorithm changes as it is trained on subsequent tasks. ### Components/Axes * **Title:** Evaluation on Task * **X-axis:** Training Sequence Per Task (T1, T2, T3, T4, T5, T6, T7, T8) * **Y-axis:** Accuracy % (Scale: 0 to 100) * **Legend (Top-Left):** * finetuning: 36.82 (22.97) - Dotted Black Line * joint*: 60.13 (n/a) - Gray Line with Triangle Markers * PackNet: 47.23 (0.00) - Green Line with X Markers * SI: 49.96 (4.33) - Orange Line * EWC: 51.09 (2.40) - Yellow Line * MAS: 50.57 (0.91) - Red Line * LwF: 47.18 (8.78) - Light Blue Line * EBLL: 47.82 (6.88) - Dark Blue Line * mean-IMM: 38.60 (19.16) - Light Brown Line * mode-IMM: 45.14 (1.87) - Dark Brown Line ### Detailed Analysis **Task 1 (T1):** * finetuning (Dotted Black): Starts at approximately 78% and drops sharply to around 52%. * joint* (Gray w/ Triangles): Starts at approximately 82% and remains relatively stable. * PackNet (Green w/ X): Starts at approximately 72% and remains relatively stable. * SI (Orange): Starts at approximately 80% and remains relatively stable. * EWC (Yellow): Starts at approximately 78% and remains relatively stable. * MAS (Red): Starts at approximately 80% and remains relatively stable. * LwF (Light Blue): Starts at approximately 74% and remains relatively stable. * EBLL (Dark Blue): Starts at approximately 76% and remains relatively stable. * mean-IMM (Light Brown): Starts at approximately 70% and remains relatively stable. * mode-IMM (Dark Brown): Starts at approximately 78% and remains relatively stable. **Task 2 (T2):** * finetuning (Dotted Black): Decreases to approximately 35%. * joint* (Gray w/ Triangles): Remains relatively stable at approximately 58%. * PackNet (Green w/ X): Remains relatively stable at approximately 52%. * SI (Orange): Remains relatively stable at approximately 54%. * EWC (Yellow): Remains relatively stable at approximately 55%. * MAS (Red): Remains relatively stable at approximately 52%. * LwF (Light Blue): Remains relatively stable at approximately 50%. * EBLL (Dark Blue): Remains relatively stable at approximately 52%. * mean-IMM (Light Brown): Remains relatively stable at approximately 50%. * mode-IMM (Dark Brown): Remains relatively stable at approximately 55%. **Task 3 (T3):** * finetuning (Dotted Black): Decreases to approximately 32%. * joint* (Gray w/ Triangles): Remains relatively stable at approximately 56%. * PackNet (Green w/ X): Remains relatively stable at approximately 48%. * SI (Orange): Remains relatively stable at approximately 48%. * EWC (Yellow): Remains relatively stable at approximately 48%. * MAS (Red): Remains relatively stable at approximately 46%. * LwF (Light Blue): Remains relatively stable at approximately 44%. * EBLL (Dark Blue): Remains relatively stable at approximately 46%. * mean-IMM (Light Brown): Remains relatively stable at approximately 42%. * mode-IMM (Dark Brown): Remains relatively stable at approximately 46%. **Task 4 (T4):** * finetuning (Dotted Black): Decreases to approximately 30%. * joint* (Gray w/ Triangles): Remains relatively stable at approximately 54%. * PackNet (Green w/ X): Remains relatively stable at approximately 46%. * SI (Orange): Remains relatively stable at approximately 46%. * EWC (Yellow): Remains relatively stable at approximately 46%. * MAS (Red): Remains relatively stable at approximately 44%. * LwF (Light Blue): Remains relatively stable at approximately 42%. * EBLL (Dark Blue): Remains relatively stable at approximately 44%. * mean-IMM (Light Brown): Remains relatively stable at approximately 38%. * mode-IMM (Dark Brown): Remains relatively stable at approximately 44%. **Task 5 (T5):** * finetuning (Dotted Black): Decreases to approximately 28%. * joint* (Gray w/ Triangles): Remains relatively stable at approximately 52%. * PackNet (Green w/ X): Remains relatively stable at approximately 44%. * SI (Orange): Remains relatively stable at approximately 44%. * EWC (Yellow): Remains relatively stable at approximately 44%. * MAS (Red): Remains relatively stable at approximately 42%. * LwF (Light Blue): Remains relatively stable at approximately 40%. * EBLL (Dark Blue): Remains relatively stable at approximately 42%. * mean-IMM (Light Brown): Remains relatively stable at approximately 34%. * mode-IMM (Dark Brown): Remains relatively stable at approximately 42%. **Task 6 (T6):** * finetuning (Dotted Black): Decreases to approximately 26%. * joint* (Gray w/ Triangles): Remains relatively stable at approximately 50%. * PackNet (Green w/ X): Remains relatively stable at approximately 42%. * SI (Orange): Remains relatively stable at approximately 42%. * EWC (Yellow): Remains relatively stable at approximately 42%. * MAS (Red): Remains relatively stable at approximately 40%. * LwF (Light Blue): Remains relatively stable at approximately 38%. * EBLL (Dark Blue): Remains relatively stable at approximately 40%. * mean-IMM (Light Brown): Remains relatively stable at approximately 32%. * mode-IMM (Dark Brown): Remains relatively stable at approximately 40%. **Task 7 (T7):** * finetuning (Dotted Black): Decreases to approximately 24%. * joint* (Gray w/ Triangles): Remains relatively stable at approximately 62%. * PackNet (Green w/ X): Remains relatively stable at approximately 38%. * SI (Orange): Remains relatively stable at approximately 38%. * EWC (Yellow): Remains relatively stable at approximately 40%. * MAS (Red): Remains relatively stable at approximately 38%. * LwF (Light Blue): Remains relatively stable at approximately 36%. * EBLL (Dark Blue): Remains relatively stable at approximately 38%. * mean-IMM (Light Brown): Remains relatively stable at approximately 30%. * mode-IMM (Dark Brown): Remains relatively stable at approximately 38%. **Task 8 (T8):** * finetuning (Dotted Black): Increases to approximately 92%. * joint* (Gray w/ Triangles): Remains relatively stable at approximately 92%. * PackNet (Green w/ X): Remains relatively stable at approximately 58%. * SI (Orange): Remains relatively stable at approximately 58%. * EWC (Yellow): Remains relatively stable at approximately 92%. * MAS (Red): Remains relatively stable at approximately 92%. * LwF (Light Blue): Remains relatively stable at approximately 92%. * EBLL (Dark Blue): Remains relatively stable at approximately 92%. * mean-IMM (Light Brown): Remains relatively stable at approximately 58%. * mode-IMM (Dark Brown): Remains relatively stable at approximately 58%. ### Key Observations * The "finetuning" algorithm (dotted black line) experiences a significant drop in accuracy after the first task and remains low for subsequent tasks until Task 8 where it spikes. * The "joint*" algorithm (gray line with triangle markers) maintains a relatively stable and higher accuracy compared to other algorithms across all tasks, except for Task 8 where most algorithms perform similarly. * Other algorithms (PackNet, SI, EWC, MAS, LwF, EBLL, mean-IMM, mode-IMM) show a gradual decrease in accuracy as the task sequence progresses, but they perform better than "finetuning" in tasks T2-T7. * In Task 8, most algorithms show a significant increase in accuracy, suggesting a potential change or reset in the task setup. ### Interpretation The chart illustrates the challenge of continual learning, where models struggle to maintain performance on previously learned tasks as they are trained on new ones. The "finetuning" algorithm suffers from catastrophic forgetting, as its accuracy drops significantly after the first task. The "joint*" algorithm, likely trained on all tasks simultaneously, provides a performance upper bound and demonstrates the potential accuracy achievable without forgetting. The other algorithms represent various strategies to mitigate forgetting, and their performance reflects the effectiveness of these strategies. The spike in accuracy for most algorithms in Task 8 suggests that this task might be significantly different or easier than the preceding tasks, or that some form of reset or adaptation occurs at this point.