Dear authors,
thanks for the submission of your paper to Climate of the Past. The paper was reviewed by two anonymous reviewers and also one public comment to your paper was submitted. All these three materials contain a lot of comments directed to improvement of the quality of your paper. From your preliminary responses to those comments it follows that there is a good potential to prepare the major revision of your paper which is requested. Since you will prepare and submit the revised manuscript including responses to all comments of all three materials, your revised paper will be again reconsidered and you will be informed later about further decisions in the matter of your paper.
I am looking forward for the major revision of your manuscript, best regards
Rudolf Brazdil

Dear Dave, dear authors,
many thanks for significantly improved revised version of your manuscript which was again evaluated by referees with respect to their original comments. There is no doubts that the paper is very valuable contribution to SI CP, but it still needs some improvements as stated in related evaluations of referees. Please respond to every of reported comments and prepare further revised version of your manuscript. I hope to be able to decide about your manuscript after obtaining of the new revised version.
With many thanks for your effort and best regards,
Rudolf Brázdil

Referee 1:
I appreciate the work that the authors have taken in revising the manuscript. Some of my comments have been taken into consideration but it is clear that we do not share the same view of what a critical review is. In their reply they say that ‘they embed critical analysis throughout the manuscript’, but sections 1-8 are still purely descriptive of the current state of the art, which, as told in my previous review is a highly valuable effort. The new version of section 9.2 has some more analytical character but limited to the treatment of the uncertainty. I think that a stronger comparative analysis among the different approaches would lead to a more convincing paper, but I do not want to go further in this controversy.
The guidelines have been partly modified following my comments, but I still have problems here.
Guideline 8. I do not agree how it is formulated. I think that a general statement advising to sum index series is misleading. Take the case of wet indices. What would happen if you add -3 (very dry period) and 3 (very wet), you will get a 0, indicating a misleading normal value, while what you actually got was two extremes within the same period. The index aggregation should be done with careful assessment, so this guideline needs to be reformulated. This comment also applies to the new text inserted in section 8 after table 1.
Guideline 10. The resolution is not the problem when calibrating indices against proxies. The problem is that proxies are an indirect indication of the climate, with their own uncertainties and caveats. They are related through a transfer function with the climate variable and they cannot be taken as the ‘truth’ for calibration. You can use them for testing and comparing but should not be considered for calibrating, since the index might have a more direct relationship with the climate variable that the proxy itself.
Other questions
The statement (lines 713-714 in the track changes manuscript) about precipitation following a gaussian distribution should be corrected. Different from temperature, the precipitation follows different distributions depending on the considered scale, from Gamma (for short periods) to gaussian. The sentence in the text should be clarified.
I do not understand the sentence in lines 924-926. “However, even where a period of overlap is lacking, indices from documentary sources can still be used to test reconstructions from proxy data or reconstruction, modelling results and observations”. I do not understand the last part of the sentence.
I am wondering why the well-known CLIWOC figure 10 is not referenced to the original papers.
One of the longest series derived from documentary sources is the Liu et al 1000 year series of typhoons in S China. This should be included in the text.
Kam-biu Liu, Caiming Shen & Kin-sheun Louie (2001) A 1,000-Year History of Typhoon Landfalls in Guangdong, Southern China, Reconstructed from Chinese Historical Documentary Records, Annals of the Association of American Geographers, 91:3, 453-464, DOI: 10.1111/0004-5608.00253

Referee 2:
I recommend to do only several minor changes in the final version of the manuscript (see below). I believe that the whole article and especially the final guideline will represent a significant contribution to the development of historical climatology.
Suggested minor changes:
Line 712: Rather there should be „…calibration and verification… “. However, as the whole paragraph is dealing with the process of index development, the last sentence may be omitted.
Line 720: “… from the mean of the reference period)”
Lines 933–934: lease consider formulation that is more correct: “During verification, index values calibrated to physical units (e.g. temperature degrees or precipitation amount) are compared with the instrumental data …”
Line 939: Please consider using e.g. “…not stable through time… ” instead of “non-stationary“. Several types of “stationarity” (and non-stationarity) may be defined strictly in statistical sense.
Line 1024–1025: There should be “… at the risk of incorporating non-homogeneities.”

Referee 3:
The manuscript has improved a lot in the review process. Nevertheless, there are some important questions than remain without solution and I cannot recommend its publication in its current form. One major point is that the authors do not accept the lack of critical analysis in the in the manuscript suggested by RC2 and SC1. In my opinion, to infer the guidelines proposed at the end of the manuscript by the previous sections continues to be impossible.
The authors are excluding of the review some indexing methods and documentary sources (see paragraph 197-203 in the manuscript, page 12 of the reply and comments below). For this reason, it is necessary to clarify on which documentary sources and type of reconstructions is focused the review. In the pag.12 of the reply letter the authors said that the narrative sources are the key focus “Space precludes a full coverage of these and comparable indices relating to Europe, so we will emphasise that our key focus is on narrative sources”. In this case I suggest including this concept clearly in the title and abstract. Moreover a clear definition of “narrative source” in the introduction is required, and modifies the manuscript according with this definition. I understand that not all the cited sources in the introduction can be considered narrative sources “Information sources include, but are not limited to, annals, chronicles, inscriptions, letters, diaries/journals (including weather diaries), newspapers, financial, legal and administrative documents, ships’ logbooks, literature, poems, songs, paintings and pictographic and epigraphic records” and many indices cited in the text are based on both narrative and no narrative sources.
I have the feeling that the authors want to focus the manuscript in situations in which the researcher has many different documentary sources with different origins and with highly variable climate information. In this situation, probably the categorized index +X/-X is a good option. If this is the idea, this must be crystal clear in the manuscript, starting by the title because currently is to general. In this case probably has no sense to include in the review the works focused on weather diaries, logbooks, phenology or chapter acts among others, because in these cases researchers use only one type of documentary sources with very specific climate information. Another option would be to divide the review in works that use many different documentary sources with high variable climate information to reconstruct one series and the works that use only one kind of documentary source with specific climate information.
In my opinion this paragraph:
line 197-203: Index series based on historical records of rogation ceremonies – closely linked to precipitation (or a lack thereof) – warrant separate discussion. This source type is particularly valuable for western Mediterranean regions (e.g. Álvarez Vázquez, 1986; Martín-Vide and Vallvé, 1995; Barriendos, 1997; Piervitali and Colacino, 2001; Domínguez-Castro et al., 2008; Barriendos, 2010; Domínguez-Castro et al., 2010; Garnier, 2010; Domínguez-Castro et al., 2012b; Fragoso et al., 2018; Tejedor et al., 2018; Gil-Guirado et al., 2019; Bravo-Paredes et al., 2020). However, as most studies base their indices on the type or cost of ceremonies – or the space within individual documents devoted to describing each ceremony – rather than a meteorological entity, we do not go into further detail.
has some conceptual errors and it is incorrect. The articles cited only have in common that they use rogation as a climate proxy. Some articles generate a precipitation index using pro pluvia and pro serenitate rogations i.e. Álvarez Vázquez, 1986; Martín-Vide and Vallvé, 1995; Barriendos, 1997; Barriendo 2010. Other as Fragoso et al. (2018) work with different climate information, not only with rogation and generate a monthly precipitation index (-1, 0, +1) and a drought record. Gil-Guirado et al. (2019) provides a drought index mainly based in pro pluvia ceremonies and an extreme rainfall index based in some pro serenitate rogations and much other climate information. Bravo-Paredes et al. (2019) analyse the use of pro pluvial rogations in Extremadura as winter NAO proxy. Finally, Piervitali and Colacino (2001), Domínguez-Castro et al. (2008, 2010, 2012b), Garnier (2010) and Tejedor et al. (2018) produce drought index from pro pluvia ceremonies information with different methodologies of indexation. For this reason, in my opinion some articles are correctly cited in section 2.3 but others have no sense in there and must be cited in section 2.5. In addition probably is better to cite the final version of Tejedor et al., 2018/19 accepted in climate of the past (Clim. Past, 15, 1647–1664, 2019, https://doi.org/10.5194/cp-15-1647-2019) instead the Climate of the Past Discussion version.
As I am mentioning, all these works provide indices of “meteorological entity” i.e. precipitation, drought or extreme rainfall events. Moreover some of these works provide original indexation methods far from the (–X,+X) continually cited in this review. In my opinion these original methodologies deserve to be commented and analysed in this review. Dominguez-Castro et al (2008) and Gil-Girado et al (2016) are methodological works, in which different indexing methods are compared. In my opinion this kind of works are necessary to know which methodologies of indexation are better, and not only try to accept and repeat the legacy of previous studies as the authors recommend in the point 4 of the guidelines.

Dear authors,
thank you for the last revised version of your paper and responses to referees. Your paper was significantly improved responding properly on many comments of referees. As specified below, the manuscript needs only minor revisions. I hope that after your last revision and responses to a referee, I will be able to accept your paper for publication in Climate of the Past.
With many thanks for your work with the manuscript and best regards,
Rudolf Brazdil

Referee comments:
The effort done by the authors to compile and summarize most of the scientific literature of the manuscript topic is highly valuable. Moreover, the article has improved a lot during the review process. However, I expected a deeper analysis of the indexing methodologies, but probably it is difficult to do all in the same manuscript. Anyway, I think that the manuscript could be published after take in consideration some minor points:

There is an incoherence among table 5 and the text. For example, table 5 indicates that there are no studies of droughts for Americas but in section 5.2 some works are cited:
Line 846 “Mendoza et al. (2007) constructed a similar series of droughts on the Yucatan Peninsula for the 16th to 19th centuries. Garza Merodio (2017) improved this index and extended it back in time (see Hernández and Garza Merodio, 2010), based on the frequency and complexity of rogation ceremonies (16th to 20th centuries)”.
Something similar occurs with the Snow/ice also in Americas, the table shows no studies but some works have been cited in section 5.3.

I have some comments to the guidelines:
Point 7 “Where reconstruction must rely on a single observer or record, or on secondary sources, appropriate levels of uncertainty should be noted in the final reconstruction (see 12)”. I agree that secondary sources must be used carefully. But I am not sure that reconstruction based on a single observer or record has more uncertainties than reconstruction with more observers. In my opinion, this statement could be in contradiction with this paragraph of the manuscript:
Line 1024-1028 “indices are compiled from a unique documentary source – such as a private diary or diaries (e.g. Brázdil et al., 2008; Adamson, 2015; Domínguez-Castro 1025 et al., 2015), a series of correspondence (e.g. Rodrigo et al., 1998; Nash and Endfield, 2002; Fernández-Fernández et al., 1026 2014) or a series of acts of municipal and ecclesiastical institutions for a location (e.g. Barriendos, 1997; Dominguez-Castro 1027 et al., 2018) – it is easier to identify and correct unexpected bias or homogeneity problems”.

Point 8 “It is advisable to sum-up index series – either in time (i.e. from monthly to seasonal or annual) or in space (i.e. by combining several index series from a climatologically homogeneous region). This approach may well approximate index series to natural climate variability. Careful assessment is needed, however, to avoid any loss of information during the process of summation, particularly for extreme events (see section 8.1)”. First, are there any evidences for this affirmation: “This approach may well approximate index series to natural climate variability?” could you provide some references?. Secondly, I agree with the authors that sum-up indices produce loss of information, but not only. In my opinion also can produce unexpected bias due to some “problems”:
- The possible seasonal bias of the documentary sources. This problem is briefly commented by the authors at line 721, but it should be remembered here, because could be an important problem when sum-up series in time.
- In many cases the indexation only generates ordinal data as the authors mention in the manuscript. The ordinal data do not have metric information. Although we label each month numerically as ‘-2’, ‘-1’, ‘0’, +1, +2 the numerals do not indicate equal intervals between levels. This could produce important caveats when we sum months. We do not know if the distant among -2 and -1 is the same than the distant among +1 and +2. Neither we know if this distance among levels is the same in the different months of the year. In my opinion this is a key point in the indexation and could be useful to be commented in section 8.

Dear authors,
thanks for the last corrections of your manuscript. Your paper is now recommended for publication in Climate of the Past - congratulations! Many thanks that you selected this journal for publication of your very interesting paper.
With best regards,
Rudolf Brázdil