Climate of the Past Discussions Interactive comment on “ HadISDH : an updated land surface specific humidity product for climate monitoring ” by K

Thank you for your support on this work. Your comments provide very useful areas for improvement. I hope you are happy with how we have addressed these – as described in line below. Changes to the text are shown in red. We have changed a few of the figures in addition to comments made by reviewers (e.g., Fig. 5 now shows the ratio of raw to homogenised trends as opposed to difference because this clearly shows gridboxes where the trend is of the same direction vs where the homogenisation has changed the direction of the trend). All amended figures are shown at the end of this report. We have also taken the opportunity to update to the end of 2012 and rectify a problem with the sea level pressure algorithm that became apparent during our revisions (see our response to your comment #7 related to the station pressure).


Introduction
Specific humidity at the surface is, on a physical basis, expected to increase commensurate with rising surface temperatures, where the presence of liquid water is not a limiting factor (Held and Soden, 2000).This has been observed over recent decades (Dai, 2006;Willett et al., 2008;Berry and Kent, 2009)  in excess of 7 % per kelvin (as expected from the Clausius-Clapeyron relation) for some regions over 1973-1999(Willett et al., 2010)).Surface water vapour drives a positive feedback effect supplying the upper atmosphere with additional water vapour through vertical mixing processes.Here it acts as a greenhouse gas modifying the radiation budget and augmenting climate change.Water vapour is also an important component of the Earth's atmosphere for a number of additional reasons beyond determining climate sensitivity.The amount of water vapour in the atmosphere, quantified here as specific humidity, is a crucial element within the hydrological cycle: it governs heavy rainfall amounts where a large fraction of the water is often rained out (Trenberth, 1999).A number of variables are now showing what appears to be an intensified hydrological cycle (e.g.precipitation - Zhou et al., 2011;ocean salinity -Durack et al., 2012;evaporation -Brutsaert and Parlange, 1998), which is consistent with largescale increasing water vapour concentration.Through latent heat, water vapour stores and releases energy, which can then be transported around the globe.Increasing water vapour also has implications for regulation of thermal comfort, increasing the risk of heat stress or heat related health problems in humans (Taylor, 2006) and impacting milk yields in cattle (e.g.Segnalini et al., 2011;Vujanac et al., 2012) amongst other physiological impacts on ecosystems more generally.Since early in the 21st Century however, humidity increases over land have abated somewhat as global land temperatures have continued to rise.This has been observed as a decrease in the relative humidity and a plateauing in the specific humidity (Simmons et al., 2010;Willett et al., 2012).Simmons et al. (2010) suggest a link to the observed greater warming over the land than over the oceans in recent years.Potential mechanisms for such warming asymmetry have been discussed in the literature (e.g.Brutsaert and Parlange, 1998;Joshi et al., 2008;Rowell and Jones, 2006).Much of the moisture over the land comes from evaporation over the oceans, so if the air over the ocean surface warms more slowly than that of the land, then the saturated vapour pressure (waterholding capacity) will also increase more slowly over the ocean.Therefore, evaporation over the oceans is unlikely to increase at a rate high enough Introduction

Conclusions References
Tables Figures

Back Close
Full to sustain constant relative humidity (and hence proportionally increasing specific humidity) over the warmer land mass.Large-scale changes in the atmospheric circulation may also play a part and reduced moisture availability over land may lead to increased partitioning of incoming energy into sensible heating as opposed to evaporation (latent heating).This further escalates the warming over land and may diminish specific humidity increases.Whatever the drivers or processes, the crucial issue is how well we can characterise the true changes in surface humidity.Without a robust estimate of the observed behaviour the potential for false conclusions or inferences is substantial.
Previously, HadCRUH, a quality controlled and homogenised global surface humidity product, has been widely used to look at these changes.However, it was last updated in 2007 and an improved version, extending spatial coverage and with capacity for annual updating, is required for near-real-time monitoring activities.Here we describe the creation of the land-surface specific humidity component of an envisaged next generation HadCRUH product: HadISDH (Met Office Hadley Centre [in collaboration with the National Oceanic and Atmospheric Administration's National Climate Data Center, the National Physical Laboratory and the Climatic Research Unit] ISD humidity product).This builds upon the new hourly land surface dataset HadISD (Dunn et al., 2012) which is a quality-controlled database of global synoptic data since 1973.Section 2 describes the source data and processing.Section 3 describes the building process including the pertinent aspects of the HadISD quality control suite and the applied homogenisation procedure.Section 4 describes the development of the uncertainty model for both the station data and the gridded product.Methods for exploring these uncertainties in following analyses are also documented.An analysis of recent changes is given in Sect. 5 followed by the logistics for using the product in Sect.6. Conclusions are drawn in Sect.7. Introduction

Conclusions References
Tables Figures

Back Close
Full HadISDH uses the global high-resolution quality controlled land surface database HadISD as its source.HadISD was designed for studying extreme events and provides hourly to six-hourly temperature (T ), dewpoint temperature (T d ), sea level pressure (SLP) and wind speed for 6103 stations.It is described fully in Dunn et al. (2012).
Elements of this processing relevant to the creation of a specific humidity dataset will be discussed here.
The station data for HadISDH are essentially the same as for HadCRUH: NOAA's National Climate Data Center's Integrated Surface Database (ISD) (Smith et al., 2011).This is available online from http://www.ncdc.noaa.gov/oa/climate/isd/index.php.For HadISDH, 3462 stations are found that have sufficient length of data after passing through the quality control and homogenisation procedures (there are 54 additional stations that are sufficiently long but not included due to homogenisation issues -Sect.3.2).In order to be able to calculate a reliable climatology, each station must have at least 15 years of data within the 1976-2005 climatology period for each month of the year, where each month must contain at least 15 days.To prevent biasing towards night or day, or biases arising from systemically changing observation times aliasing into the record, there must be at least four observations per day with at least one in each eight hour tercile (0-8 h, 8-16 h, 16-24 h) of the day.This includes 1095 stations that were not in the specific humidity land component of HadCRUH.Furthermore, a total of 458 stations are the result of compositing multiple stations where they appeared to be the same station.For example, certain countries changed their WMO identifier code leading to changes in station reporting ID over the Global Telecommunication System (GTS), which is the basis for ISD.Without such compositing many Canadian, Scandinavian and Eastern European stations would be truncated or treated as two stations artificially.Unfortunately, the compositing does not manage to resolve the WMO identifier change over Eastern Germany.Compositing was done during the HadISD processing and is fully documented therein (Dunn et al., 2012).Introduction

Conclusions References
Tables Figures

Back Close
Full HadISDH improves coverage over North America, where, for HadCRUH, many records were short and fragmented although they actually referred to the same station.ISD has been improved in this regard since the creation of HadCRUH, and the compositing process has helped further.Central Europe and islands in the Pacific are also areas of better coverage than HadCRUH.However, 876 stations from HadCRUH are no longer in HadISDH.In particular there are now very few data for Madagascar, the Arabian Peninsula, Western Australia and Indonesia.This is mostly because of the lack of up-to-date data from those stations reaching the ISD databank through the GTS.This results in station records being too short to meet the criteria set out above.Hopefully this situation will be improved in future annual updates of HadISDH.In some cases, these stations will have failed to pass the new quality control and homogenisation routines with sufficient data.In a few cases, the compositing process may have resulted in a HadCRUH station having a different identifying number (WMO identifier) in HadISDH.Station coverage, including composites, is shown in Fig. 1 and a full list is available alongside the data-product at www.metoffice.gov.uk/hadobs/hadisdh.
The quality controlled (see Sect. 3.1) HadISD T d are converted to specific humidity (q) using the same equations as for HadCRUH (Table 1 -Eqs.1-5).First, T d are converted to vapour pressure (e) using Eq. ( 1).The wet-bulb temperatures (T w ) are then calculated using Eq. ( 5).Where T w values are below zero, values of e are recalculated with respect to ice (Eq.2).This assumes that the wet-bulb was indeed an ice-bulb at that time and that the measurement was taken with a wet-bulb thermometer as opposed to a resistance or capacitance sensor.This assumption will be incorrect in some cases, especially in the later record where more automated sensors are in use.This potentially introduces a dry bias in q where resistance or capacitance sensors are used when the ambient temperature is near or below 0 • C because e calculated with respect to ice is lower than that with respect to water at the same temperature (NPL/IMC, 1996).Given the increasing propensity in the record for such measurements, unless the effects are detected and accounted for in the homogenisation this would tend to yield to a spurious drying signal in locations and seasons where sub-freezing temperatures Introduction

Conclusions References
Tables Figures

Back Close
Full are frequent.However, absolute values of specific humidity are small under such conditions so absolute errors will be small even if they are large in percentage terms.They will not affect records in seasons with temperatures above freezing.Without metadata for all 3462 stations it is impossible to correct for this and so it remains an uncertainty in the data, but it should bear little influence on the large-scale assessments for which this product is intended.From e, Eq. ( 3) is used to calculate q.As with HadCRUH, the station pressure (P ) is always derived from the standard atmosphere pressure (1013 hPa) and correction for station elevation (Z in m) by assuming a 1 hPa decrease in pressure with 10 m elevation increase: P = 1013−(Z/10).The sea-level pressure information included in HadISD is not used as it is not always simultaneously available.The potential errors from assuming a static P are very small, in the order of ∼ 0.1 % of q per 1 hPa increase in P (Willett et al., 2008).We assume that pressure will vary above and below the standard station pressure and so essentially cancel out.
3 Building the data-product

Quality control
Synoptic data contain random and systematic errors which must be removed as far as is possible to ensure robust climate analyses.The random errors can be caused by instrument error, observer error or transmission error.As part of the HadISD processing a suite of quality control tests were designed for use with hourly synoptic data.These tests have been optimised with the aim of removing random errors while retaining the "true" extremes.The quality control suite included tests particular to humidity and also neighbour intercomparisons.Full The HadISD QC comprises 14 tests which looked at 6103 stations selected from the ISD database after compositing.These tests are more sophisticated than those conducted for HadCRUH as they have been designed iteratively by validation with stations where specific problems were known or record values documented, and then further tuned to optimise test performance.Like HadCRUH, a set of three logical checks are included to test for humidity measurement failures.The first tests for supersaturation: where T d exceeds T , the T d observations are removed.If this occurs for more than 20 % of the observations within a month, the whole month is removed.The second is for occurrences of the wet-bulb wick drying out, either through reservoir drying or freezing, which again assumes the majority of humidity measurements were taken using psychrometers.This test uses dewpoint depression: if there are ≥ 4 consecutive observations spanning 24 h or more where the dewpoint depression is < 0.25 For HadISDH stations data removal is highest in the regions of greatest data density (North America and North-Western Europe) as shown in Fig. 2a, b.This is similar for both T and T d but with a higher percentage of T d data removed, especially around the Introduction

Conclusions References
Tables Figures

Back Close
Full tropics.For T d , in total 77.0 % of stations have ≤ 1 % of hourly data removed and 97.6 % of stations have ≤ 5 % of hourly data removed.For T , 89.6 % of stations have ≤ 1 % of hourly data removed and 99.2 % of stations have ≤ 5 % of hourly data removed.

Homogenisation
The monthly mean values are likely to contain systematic errors due to changes in instruments, station moves, incorrect station merges, changes in observing practices or changes to local land-usage.For this reason, the monthly mean q data are reprocessed to detect and adjust for undocumented changepoints.There are now a number of available homogenisation algorithms that have been developed and benchmarked for temperature and precipitation as part of the COST HOME project (Venema et al., 2012; www.homogenisation.org).However, very few are suitable to be run on large global networks which require an automated process.The pairwise homogenisation algorithm designed for NCDC's GHCN Monthly global surface temperature record (Menne and Williams, 2009) has been chosen here.This has been shown to be one of the more conservative algorithms, giving a very low rate of changepoint detection where none are actually present (false alarm rate) (Venema et al., 2012).Also, the pairwise method enables attribution of a changepoint to a station or stations in a more robust manner than a simple candidate verses composite reference series approach.In the candidate-composite reference series approach, network wide changes may be missed or wrongly attributed to a single station.Furthermore, the pairwise homogenisation algorithm has been through a substantive benchmarking assessment for the US temperature network (Williams et al., 2012).This showed that in all benchmark cases, the pairwise algorithm reduced the errors in the data.Importantly, it did not over-adjust or make the data any worse.This is the first time that the pairwise algorithm has been used on surface humidity data or indeed any data outside of station temperature records.
The pairwise algorithm (Menne and Williams, 2009;Williams et al., 2012) undertakes a number of sequential steps to find and adjust for suspected changepoints in the series: Introduction

Conclusions References
Tables Figures

Back Close
Full 1.For a candidate station a set of neighbours are selected based upon monthly mean time series correlation.
2. The difference series between each station and every neighbour are assessed iteratively using the standard normal homogenisation test (SNHT; Alexandersson, 1986) to locate undocumented change points.At this point both the candidate and master are tagged as potential breaks.
3. The large array of potential breakpoint locations is resolved iteratively as shown by the following overly simple illustration.A station might have 20 potential breaks assigned in close proximity and its 20 neighbours only one each.In this case it is clear that this station contains the true breakpoint.All of the changepoints are assessed together to determine the date of the changepoint.The break count for all the remaining stations is reduced by one so all then would be treated as homogeneous.
4. The changepoint is then assessed to define whether it is indeed a step-change or actually part of a local trend.Where the magnitude of a changepoint can be reliably estimated, and reasonable confidence can be assigned that it is non-zero based upon the spread of pairwise adjustment estimates arising from apparently homogeneous neighbour segments resulting from step #3, a flat adjustment is made to the mean of the homogeneous sub-period, using the most recent period as a reference.Where the magnitude of the changepoint cannot be reliably estimated, that period of data is removed.The spread of estimated changepoint magnitudes across the network also provides a 2σ estimate of uncertainty for the applied adjustment.This is fed through to the station uncertainty (Sect.3.3).
Overall, the pairwise homogenisation results in an adjustment rate of approximately two per station.The adjustments applied to HadISDH are reasonably symmetrical about zero with a median of −0.8 g kg −1 and 90 % of the adjustments lying between −0.84 and 0.81 g kg −1 (Fig. 3a).The historical spread is also relatively even (Fig. 3b).The Introduction

Conclusions References
Tables Figures

Back Close
Full first two years and last two years are artificially free from changepoints because it is more difficult to detect changepoints close to the end of record.As all adjustments made are seasonally invariant and additive rather than proportional it is highly likely that adjustments may be biased low in some seasons and biased high in others.This is a particular problem for very dry months where adjustments to already very low specific humidity results in unphysical negative values.This occurs in 54 stations (identified in Fig. 1b) although only 16 of these have more than 2 % of their data affected.For this version of HadISDH these stations will not be included in any further analyses.There may also be issues at the saturation end where positive adjustments bring the specific humidity above the saturation level imposed by the original unhomogenised temperature data.However, it is likely in many cases that any inhomogeneities appearing in specific humidity co-occur in the dry-bulb temperature, which would change the saturation level.This suggests that seasonally varying and proportional adjustments may be a better approach for specific humidity, such that the humidity can never go below 0 g kg −1 .Homogenisation of specific humidity is still a relatively new endeavour.Exactly how the different types of inhomogeneity affect the specific humidity across the seasonal cycle, or even in wet verses dry years, is not well understood.On further investigation (Fig. 4a, b), there is no obvious relationship between adjustment magnitude and climatological mean specific humidity, as shown by looking at adjustment magnitude by latitude.Adjustment direction is relatively evenly spread across 0 g kg −1 at all latitudes.Should the relationship between adjustment magnitude and specific humidity be strong, we would expect to see the largest adjustments made in the more humid tropics.In fact, the largest adjustments occur in the extratropics.Figure 4a also shows that it is easier to detect smaller changepoints in well sampled regions, as shown in Menne et al. (2009) for the USA.There is little geographical coherence in adjustment magnitude or direction, as shown by Fig. 4b.Given the complexity of seasonal adjustment magnitude, we have chosen to start with the simple approach of seasonally-invariant flat adjustments, where the transforms to the data are easily traceable, rather than making more complicated assumptions.In Introduction

Conclusions References
Tables Figures

Back Close
Full terms of detecting long-term trends in the anomalies over large spatial scales, this approach should differ very little from a seasonally-varying and proportional adjustment approach over each homogeneous subperiod.The absolute values, however, especially on gridbox spatial scales and sub-annual temporal scales, should be used with caution.
Figure 4a-c shows trends in the data before and after homogenisation.There is generally good agreement with 89.7 % of gridboxes being of the same sign (drying or moistening) in both the unhomogenised and homogenised data.However, it is clear that the unhomogenised data show trends of a greater magnitude (both wetter and dryer) than the homogenised data.A set of individual stations representing some of the largest changes in trends before and after homogenisation are displayed with respect to the surrounding network in Fig. 5a-c.
There will always be changepoints (both step-changes and local trends) which remain undetected because they are either too small to detect or too close to other changepoints.It is very difficult to estimate the uncertainty remaining in the data due to missed detections and adjustments without a rigorous benchmarking exercise as has been undertaken for temperature over the USA (Williams et al., 2012).Benchmarking is a relatively new concept and so has not yet been attempted for humidity and as such, is beyond the scope of this paper.

Station uncertainty
Our estimate of the monthly mean anomaly q anom is given, following Brohan et al. (2006), by: q anom = q ob − q clim + q adj (6) Introduction

Conclusions References
Tables Figures

Back Close
Full where q ob is the observed monthly mean; q clim is the climatological monthly mean over the 1976 to 2005 reference period and q adj are the adjustments applied to improve the long-term homogeneity.In fact, there is an error term, ε, inherent in each of these terms such that the true monthly mean anomaly can be described as: Unfortunately, these errors cannot be quantified explicitly, and so the uncertainty, u, in each monthly mean anomaly value needs to be estimated.To determine the significance of q anom we estimate the uncertainty u anom that captures the likely error from each of the error terms in Eq. ( 7): where u clim is the uncertainty in the calculation of the climatological monthly mean due to missing data (temporal sampling uncertainty); u adj is the uncertainty in the adjustments applied for homogeneity; and u ob is the measurement uncertainty of meteorological measurements.We now consider each of these in turn.
The standard uncertainty in the climatological monthly mean due to missing data is given by: where σ clim is the standard deviation of the N months making up the climatological mean of the 30-yr period from 1976 to 2005.The standard uncertainty in the applied homogeneity adjustments, u adj is estimated as the quadrature sum of two terms:

Conclusions References
Tables Figures

Back Close
Full The first term u applied arises from the adjustments which have been applied to the data, and the second term u missed arises from the adjustments which have not been applied to the data, but which should have been.
We estimate u applied from the 5th to 95th percentile spread of all possible adjustments magnitudes given by the network of pairwise evaluations, as described in Sect.3.1, adjusting by a factor 1.65 to obtain a standard uncertainty (1σ, coverage factor of k = 1).
We estimate u missed , the uncertainty arising from missed changepoints, using methods described in Brohan et al. (2006).We assume that large changepoints, shown in the tails of the distribution in Fig. 3a (black line), are well captured because they are easy to detect given the high signal-to-noise ratio.However, the small adjustments, close to 0 g kg −1 are not well captured, as shown by the "missing middle" of the distribution.The central part of the distribution can be approximated by a Gaussian distribution.A best-fit curve is then derived by merging a fitted Gaussian curve (near the centre) with those points of the actual adjustment distribution that are larger (in the wings).The standard deviation of the difference between this "best-fit" and the actual distribution (blue dotted line) is 0.15 g kg −1 and provides an estimate of u missed .
The uncertainty u ob relates to the uncertainty of measurement of the instrument at the point of observation.The BIPM "Guide to the Expression of Uncertainty in Measurement" (BIPM, 2008) describes uncertainties as belonging to one of two categories.Type A uncertainties are those which can be estimated from analysis of repeated observations.Type B uncertainties are those which cannot be estimated by repeated observations, and so must be estimated from a priori knowledge of the measurement apparatus and the measuring conditions.Type B uncertainties may have randomly varying components, u rand , and components which cause "systematic" errors, u sys .
In a meteorological context it is not possible to derive Type A estimates of uncertainty because the measurand -the weather -is intrinsically variable, and so the variability due to the instruments themselves cannot be isolated.Since Type A uncertainties are likely to be random and uncorrelated, they should reduce with temporal and spatial averaging to a large extent, and so be attenuated by averaging both over Introduction

Conclusions References
Tables Figures

Back Close
Full a month and over a gridbox.Since the station metadata do not reliably record the instrumentation used, we have derived estimates of the Type B uncertainty of an individual measurement u i based on knowledge of hygrometers in use in the field.Until the 1980s, psychrometers were probably the most common type of hygrometer, but since then there has been a move towards electronic devices (typically capacitance sensors) and dewcels which can be more readily automated.Typically electronic devices have a lower uncertainty than psychrometers and so we can conservatively estimate u i (Table 2) assuming that all humidity measurements were taken using aspirated psychrometers.
Psychrometer errors arise either from the use of an incorrect psychrometer coefficient, or from temperature errors in measurements of either the wet-bulb or dry-bulb (MOHMI, 1981).In general, these errors are not random or symmetrically-distributed and they may be correlated with other meteorological variables, such as wind speed.However, we expect that within any one month, the uncertainty of measurement for psychrometers will contain some random component, u rand , whose effect can be reduced by averaging, and a systematic component, u sys , whose effect will be unaffected by averaging.
We estimate u i with a standard uncertainty of 0.15 • C in the wet-bulb depression above 0 • C. The resulting standard uncertainty in % rh varies from 1 % rh to 3 % rh, decreasing with increasing T and increasing with decreasing RH (Humidity Guide (NPL/IMC, 1996).Although not ideal, this is done at the monthly mean resolution because the homogeneity adjustments have already been applied at the monthly mean level and so it is not possible to go back to the hourly values at this stage.The concomitant uncertainty in q is estimated from the uncertainty in % rh by calculating the change in vapour pressure, e (Eq. 3) caused by changes of ±1 standard uncertainty in % rh.
Combining this with an estimate of the saturation vapour pressure calculated from simultaneous monthly mean T (Eqs. 1 and 2), under the necessary (and in many cases incorrect) assumption that the T data are homogeneous, the resulting change in q can be estimated.The reading uncertainties of the wet-bulb and dry-bulb temperatures are Introduction

Conclusions References
Tables Figures

Back Close
Full unlikely to be biased and so we assume that the resulting uncertainty is randomly distributed.We thus estimate the random component of the uncertainty in the monthly mean as: In order to pass ISD quality control there must be at least 15 days of data for a monthly mean and at least four observations per day, implying N ≥ 60.Hence, we conservatively use N = 60 in our calculation of u rand .
There are a number of weaknesses in this approach Firstly, these non-linear conversions (Eqs.1-5) are imperfect for monthly mean data.Secondly, when the monthly value is already close to 100 % rh the addition of uncertainty in RH can then result in estimates > 100 % rh.Although it is physically possible for RH to exceed 100 % rh it is not common, nor reliably measured in operational circumstances (Makkonen and Laakso, 2005).Thirdly, errors will also be introduced because the simultaneous monthly mean T data have not been homogenised.This is due to the issue of maintaining physical continuity when homogenising across simultaneously observed variables which will be addressed in future work.False wet-bulb depressions may occur at 100 % rh, but the low-resolution conversion between humidity variables makes accurate detection of such cases impossible.However limiting the new RH (% rh + derived uncertainty in % rh) to 100 % rh can imply an unrealistically small variability.To counter this, we have set a minimum threshold for u rand of two standard deviations below the mean by examining the u rand estimates for each month for the station.All values below this threshold are assumed to be unrealistically low and are substituted with the mean value of u rand for that station.
Despite the difficulties in estimating u rand one clear feature emerges (Table 2).Although the fractional uncertainties are largest at low temperatures, the absolute values of specific humidity are low in this range -saturation vapour pressure varies by a factor 20 from 0 • C to 50 • C -and so contributions to the uncertainty in the specific humidity Introduction

Conclusions References
Tables Figures

Back Close
Full of a station will be dominated by the uncertainty during periods of high temperature and high relative humidity.
In addition to randomly-varying components, u i the uncertainty of measurement of each station will also have contributions which do not reduce on averaging, u sys .Such uncertainties arise from the limitations of the calibration, and the shortcomings of psychrometers.We have not included an explicit assessment of u sys because we consider that their effect on our estimate of q anom is likely to be small.The origin of this insensitivity can be seen by considering the case in which a changepoint is identified as occurring in the record from a particular station, and also the case in which no changepoints are identified.For example, where instruments or observing practices change or stations move, u sys will change, and so we expect that some fraction of u sys should be found during homogenisation and so should be partially accounted for in terms of u adj .Additionally, when a changepoint is found by comparison with neighbouring stations, the algorithm adjusts the target station's older data to match its newer data on the assumption that more modern measurements are likely to have lower uncertainty.Where instruments or observing practices do not change, then we can assume that u sys will be substantially unchanged.So we expect that a substantial fraction of u sys will be common to q anom and q clim .Thus when calculating q anom we can expect this fraction of the uncertainty to cancel (Eq.7).However, we note that care must be taken if the final gridded data are used to estimate absolute values of specific humidity.For such cases, the full value of u sys should be evaluated to fully capture the uncertainty.The uncertainty estimates provided alongside HadISDH will therefore be underestimates with respect to the absolute values.
The uncertainties are calculated as standard uncertainties (1σ), and then a coverage factor k = 2 is applied such that there is ∼ 95 % confidence (2σ) that these uncertainties capture the true error.As an example, the individual uncertainty components and the combined station uncertainty are shown in Fig. 7 for station 486 650 (Malacca, Malaysia, 2.267 • N, 102.250 • E, 9.0 m).Climatological uncertainty is constant year-toyear but has an annual cycle and is greatest during the season of greatest natural Introduction

Conclusions References
Tables Figures

Back Close
Full variability in q.Measurement uncertainty (not including u sys ) is usually the smallest component.It changes throughout but has a clear annual cycle due to the temperature dependence.The adjustment uncertainty is usually the largest component, reducing towards 0 g kg −1 because the most recent period is treated as the reference period.This is the first attempt at quantifying uncertainty in specific humidity and is a basis which will benefit from future improvements in the model design and application as a greater understanding of this issue accrues.

Gridding methodology and sampling uncertainty
HadISDH is intended for the purpose of studying change on large temporal and spatial scales, so gridding is essential.It reduces the effect of individual outliers and remaining random errors in the data.Given that station density is rather sparse there is little value in gridding at finer than 5 • by 5 • resolution for the majority of the globe.For station-rich regions, specific high-resolution grids could be produced but will not be presented here.Using 5 • by 5 • grids also allows comparison with other products such as HadCRUH and CRUTEM4.
Gridbox estimates use all stations within the gridbox weighted equally: there is no interpolation of information from surrounding gridboxes (Brohan et al., 2006;Jones et al., 2012).Both the absolute values and the anomalies relative to the 1976-2005 reference period are gridded in addition to the monthly climatologies calculated over the reference period.The standard deviation of all contributing stations is also given for each gridbox month, providing an estimate of gridbox variability.Where only one station contributes, an arbitrarily large standard deviation of 100 is given so that these can be easily identified.Station numbers for each gridbox month are also recorded.
Station uncertainty estimates, as defined in Sect.3.3, are also brought through to the gridbox level by assuming independence of, and combining in quadrature, all constituent station uncertainties and then multiplying by 1/ (N) where N is the number of stations in the gridbox.Figure 8a shows an example field of gridded station uncertainty for June 1980 in g kg −1 .Station uncertainty is largest around the tropics, whereas, for Introduction

Conclusions References
Tables Figures

Back Close
Full the CRUTEM3 temperature product in Brohan et al. (2006) it is largest at the poles.The largest component is by far the adjustment uncertainty, until the most recent years of the record where it tends towards zero.The measurement uncertainty is comparable to the climatological uncertainty when averaged over the gridbox scale.All are generally largest in the tropics, where station density is generally least.These uncertainties are also gridded individually.
Given that there are not infinite numbers of stations within each gridbox the gridbox value is unlikely to be the true gridbox average.Some estimate of the sampling uncertainty is necessary.Following Brohan et al. (2006), the sampling uncertainty is estimated using the method laid out in Jones et al. (1997).For gridboxes with data we first estimate the mean variance of individual stations in the gridbox, s2 i , using: where Ŝ2 is the variance of the gridbox anomalies calculated over the 1976-2005 reference period and N is the mean number of stations contributing to the gridbox mean.The last term, r, is the average inter-site correlation and is estimated using: where X is the diagonal distance across the gridbox and x is the correlation decay length between gridbox averages.Gridbox sampling uncertainty, SE 2 , is then estimated by: However, here N is the actual number of stations contributing to the gridbox in each month, giving a time varying SE 2 .The number of stations contributing to the gridbox Introduction

Conclusions References
Tables Figures

Back Close
Full Hemisphere subtropics.This reflects the large uncertainty in adjustments made to the data.For there to be confidence in any changes apparent in the data, these changes must be larger than the combined spread of uncertainty.Brohan et al. (2006) also provide a bias estimate.However, for humidity over land, no such broad scale estimates have been assessed to date.While there are likely biases locally for urbanisation and land-use changes such as increased irrigation, it is assumed here that their effect at the large 5 • by 5 • gridbox scale is small.A recent study by Asokan et al. (2010) found changes in evapotranspiration flux resulting from irrigation over the Mahanadi River Basin in India suggesting that local water use could be important in regional climate change.Further work is needed to quantify this impact for the global scale.
Another way to explore the uncertainty would be to produce plausible ensemble estimates of HadISDH, as was done for, e.g.HadCRUT4 (Morice et al., 2012) or Remote Sensing System's Microwave Sounding Unit product (Mears et al., 2011).This is the first time that a global humidity estimate has been given any measure of uncertainty.
Creating a meaningful ensemble product that enables the uncertainty model developed here and its interdependencies through the HadISDH processing chain to be more fully explored is a future aspiration.

Conclusions References
Tables Figures

Back Close
Full To explore the uncertainty in individual gridbox trends (Sect.5.2) a simple 100 member ensemble of HadISDH is created, randomly sampling across the spread of the 2σ uncertainty for each individual uncertainty component (climatology, measurement, adjustment and sampling uncertainty).This is distinct from that described at the end of Sect.4.2 which would be a far more rigorous exploration of the uncertainty fields.The ensemble members created here while they available to users, are purely for exploring the spread of uncertainty and not to be used singularly as a plausible estimate of land surface humidity.
For each station, ten versions of anomaly time series are created by adding the actual anomaly values to random values of climatology, measurement and adjustment errors.For climatology error time series, ten values are randomly selected from a Gaussian distribution (µ = 0, 2σ = 1) for each station.The Gaussian is forced to have 2σ = 1 because this then provides a ∼ 95 % chance that the randomly selected values lie between −1 and 1.These values are then used as a scaling factor on the 2σ climatology uncertainty which has an annual cycle but is constant year-to-year.For measurement error, ten time series are created by randomly selecting values from the Gaussian distribution for each station and each month.These are then used as scaling factors on the 2σ measurement uncertainty, such that the error randomly varies over time.For adjustment error, ten time series are created by randomly selecting values from the Gaussian distribution for each station and each homogeneous subperiod (the period between two adjustments).These are then used as scaling factors on the adjustment uncertainty for each homogeneous subperiod.For each station, the actual anomalies are then added to the first climatology error time series, the first measurement error time series and the first adjustment error time series to give the first station realisation.
The second to tenth station realisations are created similarly.These ten realisations of each station are then gridded in the same manner as the actual HadISDH to give ten gridded realisations.For sampling error, ten values are randomly selected from

CPD Introduction Conclusions References
Tables Figures

Back Close
Full the Gaussian distribution and used as scaling factors on the sampling uncertainty.The scaling factor is consistent across all gridboxes and months within each of the ten sampling error realisations.These are then combined with each of the 10 gridded station realisations to give the 100 member ensemble of the gridded HadISDH.
To explore the uncertainty over large-area averages (Sect.5.3) the spatial coverage uncertainty is estimated and combined with the station and sampling uncertainties, after Brohan et al. (2006) for the globe, Northern Hemisphere, tropics and Southern Hemisphere.As the spatial coverage of the gridded data is not globally complete and varies from month to month, this uncertainty needs to be accounted for when creating a regional average time series.To estimate the uncertainties of these large-area averages, which are based on incomplete coverage, we use the ERA-Interim reanalysis product.For each month in the HadISDH q anomalies, the ERA-Interim q anomalies from all matching calendar months are selected (i.e. for a January in HadISDH, all Januaries in ERA-Interim are selected).The ERA-Interim fields are then masked by the spatial coverage in HadISDH for that particular month and a cosine-weighted regional average is calculated.The residuals between these masked averages and the full regional average are then calculated.From the distribution of these residuals the standard deviation is extracted and used as the spatial coverage uncertainty for that HadISDH month in the regional time series.The sampling and station uncertainties are converted from the gridded form to an area average time series form using standard error propagation formulae.These are then combined in quadrature with the spatial coverage uncertainty to obtain the final 2σ uncertainty on the area average time series.

CPD Introduction Conclusions References
Tables Figures

Back Close
Full

Validation against other land surface humidity products
It is first necessary to assess the likely quality of HadISDH before we can use it with any confidence.This has been done by comparing gridbox decadal trends with the older HadCRUH product (Fig. 9) and global area average time series with all other existing products: HadCRUH (Willett et al., 2008); HadCRUHext (Simmons et al., 2010); Dai (Dai, 2006); and the reanalysis product ERA-Interim (from 1979 onwards) (Dee et al., 2011) (Fig. 10a).In all cases trends have been estimated using the median of pairwise slopes (Sen, 1968;Lanzante, 1996).Confidence in the trend is assigned using the 95 % confidence range in the median value.Where the intervals defined by the confidence limits are either both above zero or both below zero there is high confidence that the trend is significantly different from a zero trend.The spread of these intervals gives an estimate of confidence in the magnitude of the trends.
For the gridbox trends over the common period 1973-2003, HadISDH shows generally good agreement with HadCRUH -the key feature of widespread moistening is common to both with drying apparent in parts of the South-Western USA, South Africa, South America, Australia and New Zealand.HadCRUH shows more moistening in the tropics (e.g.Eastern Brazil, West Africa and Northern India).HadISDH shows more moistening over the USA, Southern South America and South-Western Europe.There There are large differences over the USA owing to the improvements in coverage and station compositing described in Sect.2; see also Smith et al. (2011).In HadISDH, moistening is now far more widespread over the USA.For the globally averaged annual time series there is very good agreement between all data-products both in long-term changes and inter-annual behaviour.While Dai,5155 Introduction

Conclusions References
Tables Figures

Back Close
Full HadISDH and all varieties of HadCRUH use the same source data, the methods are independent and station selection differs.ERA-Interim does ingest surface humidity data indirectly through its use for soil moisture adjustment, but also has strong constraints from the 4Dvar atmospheric model and many other data products, so it can be considered independent (Simmons et al., 2010).

Spatial patterns in long-term changes in land surface specific humidity
The spatial pattern of long-term trends over the full 1973-2011 period (Fig. 11) shows a very similar picture (widespread moistening) to that shown for 1973-2003 in Fig. 9.
There are some notable differences, with more extensive drying in parts of South America and Southwestern and Southeastern USA, and Mexico; and South Africa showing significant moistening.Overall, there is widespread moistening which is strongest across the tropics.The subtropics over the USA, South America and Australia show drying.This is consistent with the now well-observed and documented intensification of the hydrological cycle over recent decades (Allan et al., 2010).The gridbox trends range from approximately −0.1 g kg −1 to 0.3 g kg −1 per decade.This is comparable to the uncertainty ranges shown in Fig. 8.To explore the uncertainty in these trends, an ensemble of HadISDH is created with 100 members as described in Sect.4.3.Trends are fitted to each ensemble member at the gridbox scale.The 5th percentile, median and 95th percentile trends for each gridbox (assessed individually) are shown in Fig. 12a-c, respectively.Moistening remains the main feature of all three maps so the conclusion of widespread moistening appears to be robust to the quantified uncertainties, especially across the tropics, Eurasia and Northeastern North America.Drying over the Southwestern USA also appears to be significant relative to uncertainty but the extratropical drying regions show relatively large uncertainty.Introduction

Conclusions References
Tables Figures

Back Close
Full For the globe, Northern Hemisphere and tropics the uncertainty range is smaller than the overall long-term trend.Hence we can be confident in the long-term moistening signal shown in the data over these regions.The decadal trend estimates (with 95 % confidence limits in the median of the pairwise slopes) are shown to be 0.095 (0.086 to 0.105) g kg −1 per decade for the globe, 0.091 (0.08 to 0.103) g kg −1 per decade for the Northern Hemisphere and 0.147 (0.133 to 0.162) g kg −1 per decade for the tropics.
The narrow ranges of the confidence limits around the trend increases our confidence in these moistening trends.For the Southern Hemisphere, which includes the drying regions of Australia and South America, the overall signal is near-zero and insignificant at 0.008 (−0.011 to 0.028) g kg −1 .The variability and uncertainty estimates are much larger.This region has few data compared to the Northern Hemisphere, both because it is mainly ocean and because station density is lower making it harder to identify and adjust for inhomogeneities.

Analysis of interannual variability in land surface specific humidity
The strong El Ni ño events of 1998 and 2010 are clear in the year-to-year variability of the data, these two years being the moistest on record.These were also two and can be downloaded from (URL to be added).While great effort has been made to ensure high quality and long-term homogeneity of the data, all users are advised to use the uncertainty estimates and station numbers contributing to each gridbox mean where possible.Feedback is very much appreciated and future versions/annual updates will endeavour to address any issues found.Table 3 documents the fields available.

Conclusions
We have presented a new improved and updated surface specific humidity product over land, HadISDH, for the purpose of assessing long-term changes.It benefits from improved station coverage and compositing, more in-depth quality control, and more thorough and objective homogenisation.It also has uncertainties parameterised through a formal error model.HadISDH has been compared against all existing global products over their respective overlaps and shown to be in very good agreement.It is the only purely observationally based estimate that exists after 2007.This is the first time that the pairwise homogenisation algorithm has been used for surface humidity.The close agreement with existing products suggests that the pairwise algorithm is an effective tool for homogenising the surface humidity data.Further work is necessary to thoroughly assess the strengths and weaknesses of this important process using humidity benchmark data in addition to exploring seasonally varying and proportionally applied adjustments The uncertainty model could also be refined.
HadISDH shows widespread and significant moistening across the globe from 1973 to 2011.This is shown to be highly significant and robust to an assessment of uncertainties that for the first time accounts in an explicit and quantified manner for random, systematic and sampling effects on estimates of large-scale specific humidity averages.Moistening is strongest over the tropics.There are a few regions showing a spatially coherent drying signal: Central-Southern South America, South-Western USA, Florida and Mexico, and parts of Australia, all essentially in the subtropics.There is generally Introduction

Conclusions References
Tables Figures

Back Close
Full lower confidence in these signals given the spread of the trend range.However, this creates a general picture of moistening wet regions and drying dry regions, consistent with the theory of an intensified hydrological cycle resulting from a warming globe.Large-scale averages exhibit increasing trends for the globe, Northern Hemisphere and tropics, suggesting that the atmosphere above the global land surface is moister now that it was in the 1970s.The moistest year on record was 1998, followed by 2010, two strong El Ni ño years and concurrently two of the three warmest years on record.There is no discernible trend for the Southern Hemisphere where variability and uncertainties are large.
It is intended for HadISDH to be updated annually so that it can be used to monitor year-to year changes in specific humidity.Future work will deliver similar products for relative humidity, vapour pressure, wet-bulb temperature and dewpoint temperature and also the simultaneously observed temperatures.Such a suite of simultaneously derived temperature and humidity products will be a valuable addition to further our understanding of the water cycle under climate change.Introduction

Conclusions References
Tables Figures

Back Close
Full  Full   (Sen, 1968;Lanzante, 1996) method.Where intervals defined by the 95 % confidence limits on the median of the slopes are both of the same sign as the median trend presented in the gridboxes the trend is presumed to be significantly different from a zero trend.This is indicated by a black dot within the gridbox.This means that there is higher confidence in the direction of the trend, but not necessarily the magnitude.The spread of the confidence interval provides the confidence in the magnitude, these values are available online at www.metoffice.gov.uk/hadobs/hadisdh.Introduction

Conclusions References
Tables Figures

Back Close
Full Discussion Paper | Discussion Paper | Discussion Paper | Screen / Esc Printer-friendly Version Interactive Discussion Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Screen / Esc Printer-friendly Version Interactive Discussion Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Screen / Esc Printer-friendly Version Interactive Discussion Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Screen / Esc Printer-friendly Version Interactive Discussion Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Screen / Esc Printer-friendly Version Interactive Discussion Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Screen / Esc Printer-friendly Version Interactive Discussion Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Screen / Esc Printer-friendly Version Interactive Discussion Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Screen / Esc Printer-friendly Version Interactive Discussion Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Screen / Esc Printer-friendly Version Interactive Discussion Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Screen / Esc Printer-friendly Version Interactive Discussion Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Screen / Esc Printer-friendly Version Interactive Discussion Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Screen / Esc Printer-friendly Version Interactive Discussion Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | mean makes a large difference to SE 2 with a 10-fold increase in stations making SE 2 an order of magnitude smaller.Sampling uncertainties, in g kg −1 , are shown in Fig. 8b as 2σ uncertainties.The main driver of the sampling uncertainty is the standard deviation of gridbox monthly specific humidity anomalies.The sampling uncertainty and station uncertainty estimates are assumed to be independent and are combined in quadrature to provide a combined uncertainty statistic, shown for June 1980 in Fig. 8c as a percentage of June climatology.Station uncertainty is the largest component and dominates the combined uncertainty fields where there are data.The magnitude of the combined uncertainty relative to climatology is generally less than 5 % but exceeds 10 % of June climatology in parts of the Southern Screen / Esc Printer-friendly Version Interactive Discussion Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | 4.3 Using the uncertainty model to explore uncertainty in long-term trends Screen / Esc Printer-friendly Version Interactive Discussion Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | is very little overall difference with 91.4 % of HadISDH gridboxes showing moistening verses 89.4 % in HadCRUH.It is likely that HadISDH contains fewer outlying/poor quality data issues due to the improved quality control and homogenisation methods used.
Screen / Esc Printer-friendly Version Interactive Discussion Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Screen / Esc Printer-friendly Version Interactive Discussion Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | 5.3 Long-term changes in large-scale area-average land surface specific humidity of the three warmest years since 1850, the third being 2005 (Sanchez-Lugo et al., 2012).However, 2005 was not an especially moist year.For comparison, annual-mean anomalies from HadISDH for 1998, 2005 and 2010 are shown alongside the global land surface temperature product CRUTEM4 (Jones et al., 2012) in Fig. 13a-f.1998 and 2010 were strong El Ni ño years, so there were generally stronger warm anomalies in the tropics than in 2005.Although 2005 was a weak El Ni ño year, its main warm anomalies were in the Northern Hemisphere in winter (not so apparent in the annual-mean shown).Following the Clausius-Clapeyron relation, the warmer tropics have a much greater increase in moisture for a given rise in temperature, than the Discussion Paper | Discussion Paper | Discussion Paper | Screen / Esc Printer-friendly Version Interactive Discussion Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Screen / Esc Printer-friendly Version Interactive Discussion Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper |

Figure 1 .
Figure 1.Station coverage change between HadCRUH and HadISDH.a) Station coverage in HadISDH.Stations in red/pink were also in HadCRUH.Stations in blue/turquoise are new.Pink and turquoise stations are stations that are composites of more than one original source station.b) Stations from HadCRUH that are no longer in HadISDH (dark green) and HadISDH stations with subzero specific humidity issues after homogenisation that are not included in any further analyses (light green). .

Fig. 1 .Fig. 2 .Fig. 3 .Fig. 6 .Fig. 9 .
Fig. 1.Station coverage change between HadCRUH and HadISDH.(a) Station coverage in HadISDH.Stations in red/pink were also in HadCRUH.Stations in blue/turquoise are new.Pink and turquoise stations are stations that are composites of more than one original source station.(b) Stations from HadCRUH that are no longer in HadISDH (dark green) and HadISDH stations with subzero specific humidity issues after homogenisation that are not included in any further analyses (light green).
It is an automated procedure, necessitated by the large number of stations and observations.It is fully documented in Dunn et al. (2012), and the HadISDH input stations are freely available for research purposes as part of the HadISD dataset at www.metoffice.gov.uk/hadobs/hadisd.Introduction

number where flags for T and T d are linked. When checking for overly fre
quent values, T d observations coincident with flagged T observations are also flagged, and where T observations exceed WMO record values for that region, the T d values are also removed.There is also a neighbour comparison where suspect values can be removed, but also, flagged values can be recovered should they agree with unflagged neighbouring values.No such comparison was made in HadCRUH.

Table 2 .
Estimates of standard uncertainty in humidity measurements calculated in terms of equivalent psychrometer uncertainty to represent a "worst case scenario".At lower temperatures the measurement uncertainty becomes large, but the low absolute specific humidity values make only a small contribution to global estimates of specific humidity.Calculations of specific humidity used Eqs.(1-5).