## Scatter Plot: Energy Consumption vs. Date for Different Token Sizes ### Overview The image is a scatter plot showing the relationship between energy consumption (Joules) and date for different token sizes. The plot includes data points for token sizes of 128, 512, 1024, and 2048. Trend lines indicate the growth of GFLOPs (Giga Floating Point Operations per Second) for all models and for models with higher GFLOPs. ### Components/Axes * **X-axis:** Date, ranging from 2017 to 2021. * **Y-axis:** Joules, ranging from 1e-01 (0.1) to 1e+04 (10,000) on a logarithmic scale. * **Legend (Top-Left):** * **Tokens:** * Pink: 128 * Purple: 512 * Blue: 1024 * Light Blue: 2048 * **Models:** * Solid Black Line: Growth GFLOPs all models * Dashed Black Line: Growth GFLOPs of models with higher GFLOPs ### Detailed Analysis * **Token Size 128 (Pink):** * Data points are clustered around the 2020 mark, with energy consumption values ranging approximately from 0.05 to 1 Joule. * There is a single data point around 2018 with a value of approximately 0.5 Joules. * **Token Size 512 (Purple):** * One data point in 2017 at approximately 1 Joule. * One data point in 2019 at approximately 20 Joules. * **Token Size 1024 (Blue):** * One data point in 2019 at approximately 100 Joules. * One data point in 2020 at approximately 200 Joules. * **Token Size 2048 (Light Blue):** * One data point in 2020 at approximately 8000 Joules. * **Growth GFLOPs all models (Solid Black Line):** * The line is nearly horizontal, indicating a very slight increase in GFLOPs over time. * The line starts at approximately 0.7 Joules in 2017 and ends at approximately 1.2 Joules in 2021. * **Growth GFLOPs of models with higher GFLOPs (Dashed Black Line):** * The line slopes upward, indicating an increase in GFLOPs over time. * The line starts at approximately 0.01 Joules in 2017 and ends at approximately 1000 Joules in 2021. ### Key Observations * Energy consumption generally increases with token size. * The energy consumption for smaller token sizes (128) is relatively stable over time. * The energy consumption for larger token sizes (1024, 2048) shows a significant increase in later years (2020). * The growth of GFLOPs for all models is relatively flat, while the growth of GFLOPs for models with higher GFLOPs shows a significant increase over time. ### Interpretation The data suggests that as token sizes increase, the energy consumption also increases, particularly in recent years. The flat growth of GFLOPs for all models indicates that the average computational efficiency has not improved significantly over time. However, the increasing GFLOPs for models with higher GFLOPs suggests that there is a trend towards more computationally intensive models, which consume more energy. The clustering of 128 token data points around 2020 suggests that these models were more prevalent during that period. The single data points for larger token sizes indicate that these models were less common but had significantly higher energy consumption.

\n ## Scatter Plot: Energy Consumption vs. Model Size Over Time ### Overview This image presents a scatter plot illustrating the relationship between energy consumption (in Joules) and time (Date) for different model sizes (measured in Tokens). Two trend lines are overlaid to show the growth of GFLOPs for all models and for models with higher GFLOPs. The data points are color-coded based on the number of tokens. ### Components/Axes * **X-axis:** Date, ranging from approximately 2017 to 2021. * **Y-axis:** Joules, displayed on a logarithmic scale from 1e-01 to 1e+04. * **Legend:** Located in the top-right corner, defining the color-coding for the number of tokens: * Pink: 128 Tokens * Purple: 512 Tokens * Blue: 1024 Tokens * Cyan: 2048 Tokens * **Lines:** * Solid Black Line: Growth GFLOPs all models * Dashed Gray Line: Growth GFLOPs of models with higher GFLOPs ### Detailed Analysis The plot contains scattered data points representing energy consumption for different models at various dates. The solid black line represents the overall trend of GFLOPs growth for all models, while the dashed gray line represents the trend for models with higher GFLOPs. **Data Point Analysis (Approximate Values):** * **128 Tokens (Pink):** * 2017: ~0.03 Joules * 2018: ~0.05 Joules * 2019: ~0.1 Joules * 2020: ~0.08 Joules (with significant variation between ~0.02 and ~0.2 Joules) * **512 Tokens (Purple):** * 2017: ~0.5 Joules * 2018: ~0.7 Joules * 2019: ~1.5 Joules * 2020: ~1.2 Joules (with variation between ~0.5 and ~2 Joules) * **1024 Tokens (Blue):** * 2019: ~2 Joules * 2020: ~3 Joules * **2048 Tokens (Cyan):** * 2020: ~10 Joules **Trend Lines:** * **Solid Black Line (Growth GFLOPs all models):** The line is relatively flat, indicating a slow and steady growth in GFLOPs for all models. It starts at approximately 0.1 Joules in 2017 and reaches approximately 1 Joule in 2021. * **Dashed Gray Line (Growth GFLOPs of models with higher GFLOPs):** This line slopes upward more steeply than the solid black line, indicating a faster growth in GFLOPs for models with higher GFLOPs. It starts at approximately 0.05 Joules in 2017 and reaches approximately 5 Joules in 2021. ### Key Observations * Energy consumption generally increases with the number of tokens. * The growth in energy consumption for models with higher GFLOPs is significantly faster than for all models combined. * There is considerable variation in energy consumption for models with 128 and 512 tokens in 2020. * The logarithmic scale on the Y-axis emphasizes the exponential growth in energy consumption for larger models. ### Interpretation The data suggests that as models grow in size (number of tokens), their energy consumption increases. The steeper growth curve for models with higher GFLOPs indicates that more powerful models require significantly more energy. This highlights the growing energy demands of increasingly complex AI models. The variation in energy consumption for smaller models in 2020 could be due to differences in model architecture, training methods, or hardware used. The logarithmic scale is crucial for visualizing the large differences in energy consumption between smaller and larger models. The plot demonstrates a clear trade-off between model performance (GFLOPs) and energy efficiency. The divergence of the two trend lines suggests that the energy cost of increasing model complexity is accelerating. This has implications for the sustainability of AI development and the need for more energy-efficient algorithms and hardware.

## Scatter Plot with Trend Lines: AI Model Energy Consumption vs. Date ### Overview The image is a scatter plot on a logarithmic scale showing the relationship between the energy consumption (in Joules) of various AI models and their publication date. Data points are categorized by the number of tokens used, and two trend lines are overlaid to show growth patterns. ### Components/Axes * **Y-Axis:** Labeled **"Joules"**. It is a logarithmic scale with major tick marks at `1e-01`, `1e+00`, `1e+01`, `1e+02`, `1e+03`, and `1e+04`. * **X-Axis:** Labeled **"Date"**. It is a linear scale with major tick marks for the years `2017`, `2018`, `2019`, `2020`, and `2021`. * **Legend (Top-Left):** * **Tokens:** A categorical legend with four entries, each associated with a colored circle: * `128` - Pink/Magenta * `512` - Purple * `1024` - Blue * `2048` - Cyan/Light Blue * **Models:** A legend for the two trend lines: * `Growth GFLOPs all models` - Solid black line * `Growth GFLOPs of models with higher GFLOPs` - Dashed black line ### Detailed Analysis **Data Points (Approximate Values by Token Category):** * **128 Tokens (Pink/Magenta):** This group has the most data points (11), clustered primarily between 2018 and 2020. Their energy consumption is generally the lowest, with most points falling between `1e-01` and `1e+00` Joules. There is a cluster of very low points (below `1e-01`) in early 2020. * **512 Tokens (Purple):** There are 3 data points. One is in mid-2017 at approximately `1e+00` Joules. Another is in late 2019 at approximately `2e+01` Joules. The third is in mid-2020 at approximately `3e+01` Joules. * **1024 Tokens (Blue):** There are 3 data points. One is in early 2019 at approximately `4e+01` Joules. Another is in late 2019 at approximately `2e+02` Joules. The third is in mid-2020 at approximately `3e+02` Joules. * **2048 Tokens (Cyan/Light Blue):** There is a single data point in mid-2020, positioned very high on the chart at approximately `8e+03` Joules. **Trend Lines:** * **Solid Line ("Growth GFLOPs all models"):** This line shows a very shallow, nearly flat upward slope from 2017 to 2021. It starts just below `1e+00` Joules in 2017 and ends just above `1e+00` Joules in 2021, indicating minimal average growth in energy consumption across all plotted models. * **Dashed Line ("Growth GFLOPs of models with higher GFLOPs"):** This line shows a steep, exponential upward slope. It starts below `1e-01` Joules in 2017 and rises to intersect the `1e+04` Joules level by 2021. This indicates rapid growth in energy consumption for a subset of more powerful models. ### Key Observations 1. **Clear Stratification by Token Count:** There is a distinct vertical separation between the data series. Models using more tokens (1024, 2048) consistently consume orders of magnitude more energy than those using fewer tokens (128, 512). 2. **Exponential Growth for High-End Models:** The dashed trend line, which aligns with the higher token count data points (1024, 2048), demonstrates that energy consumption for top-tier models is growing exponentially over time. 3. **Stagnation for Lower-End Models:** The solid trend line, influenced heavily by the numerous 128-token models, suggests that the energy consumption for the broader set of models has remained relatively constant. 4. **Outlier:** The single 2048-token data point from mid-2020 is a significant outlier in terms of absolute energy consumption, being nearly an order of magnitude higher than the next highest point (a 1024-token model). ### Interpretation The chart illustrates a bifurcation in the development of AI models. While the average energy consumption across all models (solid line) shows little growth, this masks a dramatic divergence. A subset of models, likely those pushing the boundaries of capability (implied by "higher GFLOPs" and correlated with higher token counts), is on a path of exponentially increasing energy demands (dashed line). This suggests that scaling up model size and training data (tokens) comes at a severe and rapidly escalating energy cost. The data implies that advancements in AI efficiency are not keeping pace with the scaling of the most powerful models, leading to a significant and growing environmental footprint for cutting-edge AI research. The stark separation between token categories indicates that the number of tokens is a primary driver of energy consumption, more so than the passage of time alone.

## Line Chart: Growth in GFLOPs vs. Joules Over Time ### Overview The chart illustrates the relationship between computational growth (measured in GFLOPs) and energy consumption (measured in Joules) for AI models over time, from 2017 to 2021. Two trend lines and scattered data points represent different model configurations. ### Components/Axes - **X-axis (Date)**: Spans 2017 to 2021, with annual increments. - **Y-axis (Joules)**: Logarithmic scale ranging from 1e-01 to 1e+04. - **Legend**: Located in the top-left corner, with four color-coded categories: - Pink: 128 tokens - Purple: 512 tokens - Blue: 1024 tokens - Cyan: 2048 tokens - **Lines**: - Solid black: "Growth GFLOPs all models" - Dashed black: "Growth GFLOPs of models with higher GFLOPs" ### Detailed Analysis - **Solid Black Line (All Models)**: - Starts near 1e+00 Joules in 2017. - Gradually increases to ~1e+01 Joules by 2021. - Represents the average growth trajectory across all models. - **Dashed Black Line (Higher GFLOPs Models)**: - Begins near 1e+01 Joules in 2017. - Rises sharply to ~1e+04 Joules by 2021. - Indicates faster growth for models with higher GFLOPs. - **Data Points**: - **Pink (128 tokens)**: Clustered below the solid line, mostly between 1e-01 and 1e+00 Joules (2018–2021). - **Purple (512 tokens)**: Two points near the solid line (~1e+00 Joules) in 2017–2018, one at ~1e+01 Joules in 2019. - **Blue (1024 tokens)**: Follows the dashed line closely, with points at ~1e+02 (2019), ~1e+03 (2020), and ~1e+04 (2021). - **Cyan (2048 tokens)**: Single point at ~1e+04 Joules in 2021, aligning with the dashed line. ### Key Observations 1. **Divergence in Growth Rates**: Models with higher token counts (blue/cyan) exhibit significantly faster GFLOPs growth compared to lower token models (pink/purple). 2. **Energy Consumption Correlation**: Higher GFLOPs growth correlates with exponentially greater energy use (Joules), especially for models with 1024+ tokens. 3. **Temporal Trends**: The steepest growth occurs post-2019, with the largest models (2048 tokens) dominating by 2021. ### Interpretation The chart demonstrates that computational efficiency (GFLOPs) and energy consumption (Joules) are strongly linked, with larger models driving disproportionate increases in both metrics. The dashed line highlights that models optimized for higher GFLOPs (likely newer architectures) outpace older, smaller models in growth rate. This suggests a trend toward increasingly resource-intensive AI development, raising concerns about sustainability and accessibility. Outliers like the 2048-token model in 2021 indicate rapid advancements in model scale, potentially reflecting breakthroughs in hardware or algorithmic efficiency.