We would like to thank the reviewer for their thorough and constructive assessment. The comments have helped us to improve the manuscript. A recurring theme across several comments concerns the preprocessing pipeline and how the structured input to the classification procedure is generated from raw historical sources. We address this systematically below and refer the reader to our response to Comment 2, where we describe our general approach to this issue.

Comment 1 — Title

While we acknowledge that the term "Large Language Model" (LLM) is increasingly associated with generative, decoder-based architectures in contemporary discourse, its use in the scientific literature has not been strictly limited to this definition. Encoder-only models such as BERT and its derivatives have been widely referred to as LLMs in peer-reviewed publications, given that they are large-scale neural language models pre-trained on extensive text corpora.

Nevertheless, we appreciate that the terminology may cause confusion for some readers. To avoid ambiguity and ensure terminological precision, we have revised the title to:

"From manual classification to transformer-based language models: assessing the quality and consistency of historical convective event records"

We believe this formulation more accurately reflects the architecture employed while preserving the intended contrast between traditional manual approaches and modern deep learning methods.

Comment 2 — HISKLID / tambora.org / Self-contained nature of the article

We note that the key characteristics of the HISKLID data collection and the tambora.org research environment are already described in Section 2. However, we agree that the article would benefit from a clearer explanation of how the structured input to the classification procedure is generated from raw historical sources.

The present article deliberately focuses on one specific, well-defined step within a source evaluation workflow described in Schatz (2023): the intensity classification of convective events. This workflow comprises the following main stages: Historical-Climatological Query, Event Search, Source Characterisation, Specification - Origin, Biographical Level, Transformation, Contextualisation and Quantification, and Plausibility Check. Each stage is further decomposed into formally defined individual processing steps, each with explicit input and output domains, as documented in full in Schatz (2023). The classification procedure described in this article constitutes the Contextualisation and Quantification stage. The sole input required for this step is the normalised quotation produced by the preceding Transformation stage, which encompasses quote extraction, linguistic normalisation, temporal standardisation, and spatial georeferencing. A compact schematic figure of the workflow will be added to the revised manuscript (see Figure A in the attachment).

We will additionally acknowledge the inherent limitations of any historical corpus: the collection constitutes a sample of an unknown total population of sources, and representativeness cannot be established in the statistical sense. What can be demonstrated, and what the seasonal analysis in this study confirms, is that the corpus exhibits physically plausible and internally consistent patterns that are robust across different subsets of the data, different time windows, and independent comparative series. This consistency is the most meaningful available proxy for representativeness in a historical climatological context.

Comment 3 — Text examples / Figure 1

We agree that concrete text examples would substantially improve the accessibility and transparency of the methodology. We will include a small selection of representative quotations in the main text, presented in three columns: original historical language, normalised German, and English translation. These examples will illustrate the linguistic variability of the corpus, spanning Middle High German to Early New High German, and demonstrate how the classification procedure is applied in practice, including the assignment of evidence classes (C1 to C3) and intensity levels. An annotated example is provided at the end of this response document (see Annotated Example).

We also agree that Figure 1 requires more contextualisation and will expand the accompanying text accordingly. The documentation of authors's occupational backgrounds serves two complementary purposes. First, it enables a source-critical assessment of potential reporting biases: different professional groups observe and describe weather phenomena through different lenses. As demonstrated in Section 4.3, the chi-square analysis confirms that source type significantly influences intensity classification (e.g., chronicles overrepresenting severe events, weather records overrepresenting mild ones). The occupational profile of authors is directly linked to the type of document they produced, and therefore to these systematic biases.

Second, the occupational metadata establishes a basis for comparison with early instrumental observation networks of the 18th and 19th centuries. Early meteorological stations were frequently staffed not by professional meteorologists but by educated laypeople in civic roles, such as rectors, engineers, and works supervisors, as documented for example in the Physikatsberichte of 1869, the Lamont station network, and the meteorological stations of the Kingdom of Bavaria. The occupational spectrum documented in Figure 1 reflects a structurally comparable observer community, suggesting a degree of methodological continuity between pre-instrumental historical records and early systematic observation networks.

Comment 4 — Manual normalisation

Normalisation focused on lexical and orthographic standardisation rather than grammatical modernisation. Idiomatic expressions were retained in their original form to preserve semantic authenticity. Out-of-fashion or dialect-specific vocabulary was resolved using the Worterbuchnetz (covering Early New High German and New High German), with additional cross-referencing of sources from comparable dialectal and temporal contexts where individual terms remained ambiguous. Semantic shifts in weather-related terminology were addressed carefully, ensuring that terms were interpreted according to their period-specific meaning rather than their modern equivalent.

For temporal references, dating was standardised using Grotefend's reference work on historical calendars, including saints' days, feast days, and semi-automated calculation of moveable feasts such as Easter. Historical place names were resolved against GeoNames to enable spatial georeferencing.

Quality assurance was ensured through iterative cross-checking of key event vocabulary, covering the core terms used for thunderstorms, thunder, hail, wind, snow and rain, against the compiled silver label lists. Problematic cases were rare and were resolved through systematic source comparison. No grammatical normalisation was applied, as the classification procedure operates at the lexical-semantic level and does not require syntactic standardisation.

These preprocessing steps correspond to the Transformation stage of the workflow described in Schatz (2023) and, as outlined in our response to Comment 2, will be made explicit in the compact workflow figure (see Figure A) to be added to the revised manuscript.

Comment 5 — Ehrmanntraut (2025)

While we indeed cite Ehrmanntraut (2025) as a methodological reference for transformer-based text normalization, the system was not employed for the automatic normalization of our corpus for the following reasons.

The system proposed by Ehrmanntraut (2025) was specifically trained on the DTA EvalCorpus, a parallel corpus of German literary texts (novels, theater plays, novellas, etc.) from c. 1780 to 1901. Our corpus, in contrast, consists of historical records of convective weather events, a highly domain-specific register that differs substantially from literary prose in both vocabulary and linguistic structure. Domain mismatch of this kind is well-known to degrade normalization performance, as Ehrmanntraut (2025) himself acknowledges when discussing the generalizability of such systems to other registers and datasets.

We therefore conducted the normalization manually, which allowed us to ensure terminological consistency for the domain-specific vocabulary present in our records. The citation of Ehrmanntraut (2025) was intended to situate our approach within the broader methodological literature on historical text normalization, and to confirm that transformer-based normalization has been validated for comparable historical German material.

Furthermore, our normalisation task extended beyond purely linguistic standardisation to encompass temporal dating and spatial georeferencing, both of which are indispensable for historical climatological analysis but lie outside the scope of systems such as Ehrmanntraut (2025). As outlined in our response to Comment 2, these dimensions will be made explicit in the compact workflow figure (see Figure A) to be added to the revised manuscript.

A further practical limitation is that a substantial portion of our corpus predates 1700, which lies outside the temporal coverage of the DTA EvalCorpus on which Ehrmanntraut (2025) was trained. Applying an automated normalisation system to part of the corpus while normalising the remainder manually would introduce systematic inconsistencies in the lexical basis, which would in turn compromise the reliability of the silver label matching and the subsequent classification. A uniform manual approach was therefore not only preferable but necessary to ensure corpus-wide consistency.

Comment 6 — Figure 2 (events per source)

We thank the reviewer for this suggestion. However, a normalised representation of events per source is not straightforward in this corpus, as many sources are compilations or critical editions that aggregate observations spanning multiple years, decades, or even centuries. A simple events-per-source ratio would therefore be misleading rather than informative, as it would conflate fundamentally different types of sources with very different temporal scopes.

Furthermore, the distribution of event density over time is already addressed in the manuscript. As Figure 9 clearly shows, the density of documented events increases progressively over the centuries, which is consistent with the growth in source availability documented in Figure 2. The calendar heat maps in Figure 9 provide a more meaningful representation of this temporal distribution than a normalised events-per-source ratio would.

As an alternative, we could provide a breakdown of event counts by language stage (Middle High German, Early New High German, New High German) or by century, which would offer additional context on the temporal distribution of the corpus without the interpretive problems associated with a per-source normalisation. We will add this information either as a supplementary panel to Figure 2 or as a brief descriptive statement in Section 2.

Comment 7 — Workflow and classification procedure

The present article focuses exclusively on the Contextualisation and Quantification stage of a source evaluation workflow described in Schatz (2023), of which the intensity classification of convective events is the central component. The figure presents a simplified overview of the main stages of the workflow as applied in this study (see Figure X). The order of individual processing steps may vary depending on the characteristics of the source. The underlying theoretical framework, including the full formalisation of individual processing steps and their ordering conditions, is documented in Schatz (2023).

The sole input to the classification step described in this article is the normalised quotation produced by the preceding Transformation stage. The compact schematic figure (see Figure X) will also clarify the relationship between the workflow described in Schatz (2023) and the multi-stage classification process described at line 87, which refers to the classification sub-process within the Contextualisation and Quantification stage.

Comment 8 — Silver Labels / inductive development

We respectfully note that this question is already addressed in considerable detail in the manuscript. The vocabularies, referred to as Silver Labels, are not unstructured word lists but formally defined, rule-based classification resources derived systematically from the corpus. Their structure, including the three evidence classes (C1: direct and specific, C2: indirect, C3: relative/qualitative), is described in Section 3.2, and the complete vocabularies including all recorded variants and regular expression patterns are documented in full in Appendices B1 to B4.

The inductive development refers to the fact that candidate terms were identified directly from the historical source material rather than imposed from external sources, ensuring domain specificity. Iterative checking means that the vocabulary was refined through repeated application to the corpus, with ambiguous or borderline cases resolved through cross-referencing with sources from comparable dialectal and temporal contexts.

Regarding subjectivity and ambiguity, we refer the reviewer to our general response on subjectivity below, which addresses this concern more fundamentally. A concrete annotated example is additionally provided at the end of this document (see Annotated Example).

Comment 9 — Test dataset / data availability

We respectfully note that both points raised in this comment are already addressed in the manuscript. The constitution and sampling of the test dataset are described in Section 3.5: the dataset was divided into training, validation and test portions in a ratio of 70:10:20, with stratified splitting to preserve class distributions.

Regarding data and model availability, we refer the reviewer to page 24, lines 418ff., where all relevant resources are explicitly referenced and linked.

On the question of subjectivity

We would like to address the recurring concern about subjectivity in a more fundamental way. Although the term "subjectivity" is often used in the context of text-based classification, we argue that it requires more precise differentiation. Any transition from natural language to quantified categories presupposes linguistic understanding and contextual interpretation. This applies to manual classification, rule-based NLP systems and probabilistic modelling approaches equally. The relevant question is therefore not whether interpretation is involved, but whether that interpretation is transparent, operationally defined and reproducible.

In text-based research, complete objectivity is neither achievable nor a meaningful standard. A more appropriate benchmark is intersubjective traceability, defined as the extent to which interpretive decisions are made explicit, documented, and open to scrutiny. The formalisation developed in Schatz (2023) meets this standard precisely by decomposing the source evaluation workflow into formally defined processing steps with explicit input and output domains, making interpretive decisions visible and traceable rather than leaving them implicit.

It is also worth noting that quantitative approaches are not exempt from this principle. Feature selection, model architecture, and parameter choices all involve interpretive decisions that are frequently less explicitly documented than the classification procedure described here. This applies in particular to approaches that treat words or phrases as context-independent units, an assumption that is difficult to sustain for historical texts, where meaning is strongly tied to linguistic register, temporal context, and idiomatic usage.

Crucially, words and phrases in historical sources can only be understood in context. This principle motivated both the design of the evidence classes (C1 to C3) and the choice of a transformer-based language model, which captures semantic relationships within a given context rather than treating words as independent of their surroundings.

Ultimately, the criterion for any classification procedure is not the absence of interpretation but agreement with a validated standard. The high classification performance, stable seasonal patterns and consistent intensity hierarchies demonstrated in this study provide empirical confirmation that the formalised, context-sensitive approach yields reproducible, scientifically valid results, which we consider the most compelling response to concerns about subjectivity.

To further illustrate the transparency and reproducibility of the classification procedure, we include an annotated text example below (see Annotated Example), showing explicitly how individual terms are assigned to evidence classes and intensity levels based on formally defined rules. Ambiguous cases are explicitly flagged rather than resolved arbitrarily, demonstrating that the procedure meets the standard of intersubjective traceability described above.

Annotated Example: Cologne, 25 August (historical newspaper)

Source text (normalised German):

Köln, vom 25. August. Vergangenen Freitag hat man in dieser Gegend ein schreckliches Donnerwetter gehabt [...] ist zugleich ein Wolkenbruch gewesen, dadurch den Dörfer Schevren und Häuser ins Wasser gesetzt worden [...] In der Gegend Woringen seien Hagelstein eines Ei groß gefallen, alle Fenstern, Hafer völlig zerschlagen. Zu Hurt hat er Wind die Bäume aus der Erden gerissen, auch der Donnerschlag im bergischen Land 30 hin- und wiedergelegene Häuser in Brand geschlagen [...] Zu Würzburg hat das Donnerwetter das Magazin- oder Munitionshaus auf dem Schlossberg aus dem Grund geschlagen, alle Bäume mit den Wurzeln ausgerissen, die umliegenden Weinberge und Felder ganz ruiniert [...]

Annotation overview:

* Note: Weinberge und Felder ganz ruiniert refers to the Würzburg thunderstorm event. The cause (hail vs. wind/lightning) is not unambiguous from the text; however, the pattern ruiniert is automatically classified as Hail C3 based on the silver label rules. This case illustrates how the procedure handles ambiguity transparently: the rule is applied consistently, and the uncertainty is documented explicitly.

This example demonstrates that each classification decision is traceable to a formally defined rule. The silver label lists (Appendices B1-B4) document all patterns, regular expressions, and word lists used. Ambiguous cases are flagged and explained rather than silently resolved, ensuring full transparency of the classification process.

We thank the reviewer for their careful and constructive assessment and for the positive evaluation of the methodological approach.

The concerns raised — ground truth definition, circular validation, treatment of uncertainty, and source-related biases — are addressed in detail in our responses to Reviewer 1, to which we refer for the full argumentation. In summary: the evidence classes (C1–C3) provide an explicit, rule-based treatment of uncertainty; source-related biases are quantified and discussed in Section 4.3 using chi-square analysis and Cramér's V; and the question of subjectivity and intersubjective traceability is addressed in a dedicated section of the reviewer responses.

Regarding the concern about circular validation specifically: the manual classifications used for fine-tuning the language model and the statistical validation of seasonal patterns are based on the same corpus, but address fundamentally different aspects of the data. The seasonal analysis validates the physical plausibility of the corpus as a whole — it does not validate the model output. The model is evaluated exclusively on a held-out test set (20% of the labelled data, stratified, not used during training), ensuring that the classification performance reported reflects genuine generalisation rather than overfitting. The two validation steps are therefore methodologically independent.