## Diagram: Vision Transformer (ViT) to Interpretable Rule Generation Pipeline ### Overview The image is a technical flowchart illustrating a machine learning pipeline that transforms an input image into human-interpretable logical rules. The process uses a Vision Transformer (ViT) to extract features, binarizes neuron activations, generates raw rules via an algorithm called "FOLD-SE-M," and finally applies "Semantic Labelling" to produce a human-readable rule set. The diagram is divided into three main horizontal sections: the input and feature extraction (left), the binarization and rule generation (top-right), and the two resulting rule sets (bottom). ### Components/Axes The diagram contains the following labeled components and text elements, organized by spatial region: **1. Input & Feature Extraction (Left Side):** * **Image:** A photograph of a bedroom interior (bed, nightstand, chandelier). * **Label:** `INPUT TO ViT` (with an arrow pointing right). * **Model Block:** An orange trapezoid labeled `ViT`. * **Output Token:** A vertical stack of orange rectangles labeled `[CLS]`. * **Layer Label:** `Sparse Concept Layer` (below the neuron circles). * **Neuron Circles:** A vertical column of orange circles labeled `n1`, `n2`, `n3`, `n4`, `...`, `nd`. * **Activation Values:** Arrows from each neuron to a numerical value: `n1 → 0.8`, `n2 → 0.1`, `n3 → 0.0`, `n4 → 0.9`, `...`, `nd → 0.2`. **2. Binarization & Rule Generation (Top-Right):** * **Process Label:** `BINARIZATION OF NEURON OUTPUTS` (with an arrow pointing right). * **Binarization Table:** A grid with an orange header row (`n1`, `n2`, `n3`, `...`, `nd`) and five data rows containing binary values (0s and 1s). * Row 1: `0`, `0`, `1`, `...`, `1` * Row 2: `1`, `0`, `1`, `...`, `1` * Row 3: `1`, `1`, `0`, `...`, `1` * Row 4: `...`, `...`, `...`, `...`, `...` * Row 5: `1`, `0`, `0`, `...`, `0` * **Algorithm Label:** `FOLD-SE-M RULE GENERATION` (with a downward arrow pointing to the Raw Rule-Set). **3. Rule Sets (Bottom):** * **Raw Rule-Set Box (Right):** A beige rounded rectangle titled `RAW RULE-SET`. It contains three numbered logical rules in a Prolog-like syntax. Key elements are color-coded (pink for `n5`, `n2`, `n41`; blue for `n1`). 1. `target(X, 'bedroom') :- n5(X), n1(X), not ab1(X).` 2. `target(X, 'kitchen') :- n2(X), not ab2(X).` 3. `ab1(X) :- n41(X), not ab2(X).` 4. `...` * **Semantic Labelling Arrow:** A leftward arrow labeled `SEMANTIC LABELLING` connecting the Raw Rule-Set to the Labelled Rule-Set. * **Labelled Rule-Set Box (Left):** A beige rounded rectangle titled `LABELLED RULE-SET`. It contains the semantically labeled version of the rules. Key elements are color-coded (red for `bed1`, `cabinet1`; blue for `pillow1`; purple for `sink1`). 1. `target(X, 'bedroom') :- bed1(X), pillow1(X), not ab1(X).` 2. `target(X, 'kitchen') :- cabinet1(X), not ab2(X).` 3. `ab1(X) :- sink1(X), not ab2(X).` 4. `...` ### Content Details The pipeline details are as follows: 1. **Input Processing:** An image of a bedroom is fed into a Vision Transformer (ViT). The ViT processes the image and outputs a `[CLS]` token representation. 2. **Sparse Concept Activation:** The `[CLS]` token activates a "Sparse Concept Layer" consisting of neurons `n1` through `nd`. The example shows specific activation strengths: `n1=0.8`, `n2=0.1`, `n3=0.0`, `n4=0.9`, `nd=0.2`. 3. **Binarization:** The continuous activation values are converted into a binary matrix. Each row in the table likely represents a different input sample or data point. The columns correspond to neurons `n1` to `nd`. A value of `1` indicates the neuron is active (above a threshold), and `0` indicates it is inactive. The first row shows neurons `n3` and `nd` are active for that sample. 4. **Rule Generation:** The "FOLD-SE-M" algorithm processes the binarized data to generate a `RAW RULE-SET`. The rules use the abstract neuron identifiers (`n5`, `n1`, `n2`, `n41`) as predicates. For example, Rule 1 states that for an input `X` to be classified as a 'bedroom', neurons `n5` and `n1` must be active, and the abnormality condition `ab1` must not hold. 5. **Semantic Labelling:** The abstract neuron identifiers in the raw rules are mapped to human-interpretable concept names (e.g., `n5` → `bed1`, `n1` → `pillow1`, `n2` → `cabinet1`, `n41` → `sink1`). This produces the final `LABELLED RULE-SET`. The rules now read as intuitive logical statements: e.g., a 'bedroom' is defined by the presence of a bed and a pillow, and the absence of a specific abnormality (`ab1`), which itself is defined by the presence of a sink. ### Key Observations * **Color Coding:** Colors are used systematically to trace concepts. Orange represents the ViT and its neurons. In the rule sets, pink highlights raw neuron IDs, while red, blue, and purple highlight their corresponding semantic labels (`bed1`, `pillow1`, `sink1`). * **Rule Structure:** The rules follow a Definite Clause Grammar (DCG) or Prolog-like syntax, using `:-` for implication and `not` for negation. They include both target classification rules and intermediate "abnormality" (`ab`) definition rules. * **Sparsity:** The term "Sparse Concept Layer" and the binarization step suggest the system aims to identify a small, discrete set of meaningful features from the high-dimensional ViT output. * **Transformation:** The core transformation is from subsymbolic, continuous neural activations (`0.8`, `0.1`) to symbolic, discrete logical rules (`bed1(X), pillow1(X)`). ### Interpretation This diagram depicts a method for **extracting interpretable, logical knowledge from a black-box vision model (ViT)**. The pipeline bridges the gap between connectionist AI (neural networks) and symbolic AI (logic rules). * **What it demonstrates:** It shows a concrete workflow for "opening the black box." Instead of just getting a classification ("bedroom"), the system produces an explicit, auditable reason: "because a bed and a pillow are present, and a sink is not present (which would indicate a kitchen)." * **Relationship between elements:** The ViT acts as a powerful feature extractor. The Sparse Concept Layer and binarization act as a bottleneck to distill these features into discrete, switch-like concepts. The FOLD-SE-M algorithm then performs rule induction from these binary patterns. Finally, semantic labelling grounds these abstract patterns in human-understandable vocabulary, likely using an external knowledge base or embedding similarity. * **Significance:** This approach addresses key criticisms of deep learning—lack of transparency and interpretability. The resulting rules can be inspected, verified, and potentially edited by humans. The inclusion of "abnormality" rules (`ab1`, `ab2`) suggests the system can also model and reason about exceptions or confounding factors. The pipeline is valuable for domains requiring explainable AI, such as medical diagnosis or autonomous systems, where understanding the "why" behind a decision is crucial.