## Diagram: Speech Processing and Protoword Discovery Flowchart
### Overview
The image is a black-and-white flowchart diagram illustrating a cyclical process for discovering spoken terms and building a lexicon of protowords from raw speech signals. The diagram uses rectangular boxes to represent processing stages, a waveform graphic to represent input, and arrows (both solid and dashed) to indicate the flow of information and feedback loops.
### Components/Axes
The diagram consists of the following labeled components and connections, arranged in a roughly vertical flow with a feedback loop:
1. **Input (Bottom Center):** A graphical representation of a speech waveform.
2. **Box 1 (Center Bottom):** Labeled `speech coding`.
3. **Arrow 1 (Center):** A solid arrow points upward from `speech coding` to the label `speech features`.
4. **Label (Center):** The text `speech features` acts as a junction point.
5. **Box 2 (Center Right):** Labeled `Siamese DNN`.
6. **Arrow 2 (Center Right):** A solid arrow points from `speech features` to `Siamese DNN`.
7. **Label (Center, above Siamese DNN):** The text `proto-phonemes` with an arrow pointing upward from `Siamese DNN`.
8. **Box 3 (Center Left):** Labeled `Spoken Term Discovery`.
9. **Arrow 3 (Center Left):** A solid arrow points from `speech features` to `Spoken Term Discovery`.
10. **Dashed Arrow (Center):** A dashed arrow points from `proto-phonemes` to `Spoken Term Discovery`.
11. **Oval (Top Center):** Labeled `lexicon of protowords`.
12. **Arrow 4 (Top Left):** A solid arrow points from `Spoken Term Discovery` upward to `lexicon of protowords`.
13. **Arrow 5 (Top Right):** A large, curved solid arrow points from `lexicon of protowords` downward to `Siamese DNN`.
### Detailed Analysis
The diagram depicts a multi-stage process with a clear directional flow and a feedback mechanism:
* **Process Initiation:** The process begins with a raw speech waveform at the bottom.
* **Stage 1 - Coding:** The waveform is processed by `speech coding`, which outputs `speech features`.
* **Stage 2 - Feature Branching:** The `speech features` are fed into two parallel pathways:
* **Path A (Right):** Features go to a `Siamese DNN` (Deep Neural Network), which produces `proto-phonemes`.
* **Path B (Left):** Features go directly to the `Spoken Term Discovery` module.
* **Stage 3 - Integration and Discovery:** The `Spoken Term Discovery` module integrates information from two sources: the direct `speech features` and the `proto-phonemes` (indicated by the dashed arrow).
* **Stage 4 - Lexicon Formation:** The output of `Spoken Term Discovery` is used to build or update a `lexicon of protowords`.
* **Feedback Loop:** The established `lexicon of protowords` feeds back into the `Siamese DNN`, suggesting the lexicon informs or refines the phoneme discovery process in subsequent iterations.
### Key Observations
* **Cyclical Process:** The most prominent feature is the closed loop between the `lexicon of protowords` and the `Siamese DNN`, indicating an iterative or learning system.
* **Dual Input to Discovery:** The `Spoken Term Discovery` box is the only component with two distinct input arrows (one solid, one dashed), highlighting its role as an integrator.
* **Dashed Line Significance:** The dashed arrow from `proto-phonemes` to `Spoken Term Discovery` may indicate a secondary, auxiliary, or probabilistic input compared to the primary flow of speech features.
* **Hierarchical Flow:** The overall flow is bottom-up (from raw signal to abstract lexicon), with a top-down feedback component.
### Interpretation
This diagram models a computational system for **unsupervised or semi-supervised learning of speech units**. It suggests a method where a machine learning model (`Siamese DNN`) first learns to extract sub-word units (`proto-phonemes`) from acoustic features. These units, along with the raw features, are then used by a separate algorithm (`Spoken Term Discovery`) to identify recurring patterns or "words" in the speech stream, forming a `lexicon of protowords`.
The critical insight is the **feedback loop**: the emerging lexicon is not just an output but also a training signal or prior knowledge that improves the `Siamese DNN`'s ability to discern meaningful phonemic contrasts in the next iteration. This mimics a developmental process where an infant's growing vocabulary helps them better perceive and categorize the sounds of their language. The system appears designed to bootstrap language acquisition from raw audio without pre-defined transcriptions.
