Post Reply  
 
 
Thread Tools Search this Thread
01-04-2021, 16:10   #16
Oneiric 3
Registered User
 
Join Date: Jan 2013
Posts: 5,119
Quote:
Originally Posted by M.T. Cranium View Post
Those co-ordinates are not carved in stone, what's the grid tolerance, nearest degree or 0.1 degree? The exact lat long as given in the EC source is 43.67N and 79.4W.

What exactly is re-analysis, estimates from maps? The maps from before 1945 would have no upper air measurements and any recreated maps showing that are based partly on actual temperatures; my data base has probably the only actual temperatures near the grid point before the airport opened in 1938, so really this is just a sort of estimate of what my data base applied to estimated maps comes out as after the fact. But I do think it correlates at a very high level from what I checked so far, mainly the January month by month and the more extreme values of other months, don't see any outliers. It's all a bit on the cold side of the actual or even urban-adjusted numbers.
I've attached data from the 'ERA 4th Generation' for the coordinates you specified for Toronto in an early post. The data only goes from 1950 up to the present year (on a monthly basis) and the resolution is stated as being 0.5 by 0.5 degrees.

And I really wish this platform would sort out its idioitic attachment system. Far too fiddly and unnecessarily complicated for my already very limited patience to endure.
Attached Files
File Type: xlsx era5-0p5deg_195.xlsx (30.9 KB, 1 views)
Oneiric 3 is offline  
Advertisement
01-04-2021, 21:52   #17
Oneiric 3
Registered User
 
Join Date: Jan 2013
Posts: 5,119
M.T. I was looking through your Toronto data sheet and just want to confirm something with you, and that is, is this section here there actual 'raw' daily readings?



I just want to be sure before I tidy it up (those God foresaken leap days should be completely abolished imo) and run it through some code to compare it with the data I posted up above just earlier.
Oneiric 3 is offline  
03-04-2021, 21:29   #18
M.T. Cranium
Registered User
 
Join Date: Apr 2007
Posts: 12,014
Yes those are raw daily means, but the monthly averages derived from them can be found in another part of the spreadsheet, they were averaged by EC and should all be within 0.1 rounded of what you would get from the work on those daily numbers. Also my tables for Toronto run top to bottom without a space for missing 29 Feb (the NYC tables have that feature) so that formulae for some columns would be different than others. Here's the grid references (range) for the monthly averages in the spreadsheet.

They are located in rows 1840 to 2021 to make it easier to figure out to which year the data belong. The columns in play are HW to II. That gives you Jan to Dec and annual means. Unfortunately I did not block the data to convert to one decimal for round numbers, e.g. you'll see April 1842 says 7 which is 7.0.

Let me know if you find this data section and have any questions. The website data are derived from their (EC) averages of monthly mean max and mean min rather than directly from averages of mean daily. It won't make much difference but some of these data would be 0.1 different from what you would get by taking the average of daily means. Once again, my table of daily means is derived from their conversions of the original F data whereas the tables above (the ones that are in F deg) are calculated by me from the entries in the original grids.

If you still wanted to average my data rather than using this table, be aware then that each column runs unbroken from 1 Jan to 31 Dec so that leap years will run from (a) days 1232-1260 for February and (b) days 1261-1291 for March compared to 1260-1290 for non-leap years (incl 1900), etc to end of columns.

My averages at the end of each row for dates have the formulae worked out for that. I suppose I should standardize this work with the NYC tables which I altered for missing leap year dates so that all similar dates of the year after Feb 28 are in the same row. Maybe the next version will have that built in. I'm nervous about throwing off formulae although they should convert along with the shifts.

Also minor point but the -7 and -3 for Jan and Feb 1840 are just rough guesses based on data from Providence RI, there was no weather station at Toronto until 1st of March 1840.

By the way, if you're transcribing these numbers into an excel file, then they also exist in print in the net-weather thread, in the second post of the thread.

You asked if they were raw data and the answer is yes, including those monthly means I mentioned.

For comparison with other data sets, I would recommend applying the urban heat island correction as follows: data 1840 to 1880 is okay as is. Data 1881 to 1890 reduce by 0.1 (to colder values), data 1891 to 1900 reduce 0.2, etc until you reach 1971 to 1980 which would reduce by 1.0 (C deg). After that 1981 to present all reduce by 1.1 C.

If it would save you some work, I could post an excel file of those conversions, that does not yet exist in the excel file but it does exist in print in the net-weather thread, this time on page two after the data on dry spells is tabulated (go to page two of the thread, second post).

If the purpose is to compare with CET or Irish data, I would prefer that these adjusted values would be compared, otherwise there will appear to be a spurious warming trend in the Toronto data over the 20th century into recent decades. If it's to compare with the re-analysis, I don't know what to recommend because I don't know if they factored into their numbers any result of the change in the environment from small city (in 1900) to metropolis (after 1960). By 1900 Toronto was already about a quarter million people with the weather station near the centre of the core, so an urban heat island would already be in place then. It would have grown approximately as suggested by my corrections thereafter. By about 1960 Toronto had surpassed two million population and had spread out a good 20-25 miles beyond its 1900 extent. Changes since then have been relatively minor with most of the growth out beyond the 1960 suburban belt, and tending to cluster rather than run out unbroken, due to greenbelt and river valley intrusions. So I don't think the urban heat island grew very much after about 1950-1960 and the data suggest that also. I used to run my own backyard weather station in a perfect spot for urban-rural comparisons (30 miles west of the downtown station) so I had about ten years of data for comparison, although the location where Toronto downtown is located would be a bit warmer than my location anyway due to lower elevation and closer proximity in winter to Lake Ontario. A sample of some very cold clear night readings would average about 6-8 F deg colder at the MTC-jr location. And that was in a small heat island of a smaller town, really rural settings might be another 4-6 F deg colder again. Most daytime readings were 1-2 deg lower there. However that was complicated by cooling lake breezes that would be just about extinct at my location inland, so some days in March to June in particular would run cooler in the city. It looked to me like that season extended into July more frequently in the 1840s and 1850s before there was any real urban development to weaken the lake breezes (and probably after some of those winters the lake was pretty cold into the middle of summer too).

Last edited by M.T. Cranium; 03-04-2021 at 21:44.
M.T. Cranium is offline  
Thanks from:
03-04-2021, 21:52   #19
M.T. Cranium
Registered User
 
Join Date: Apr 2007
Posts: 12,014
By the way the main reason why I didn't convert daily data to the same system as the monthly/annual data would be that I don't believe the differentials would apply unbroken to all days, the cool, cloudy and wet days would be reduced less than the warm, dry days especially those in modified polar air masses in warm seasons. So it would be a complicated process to change all the daily data. It would probably result in a more variable adjustment with the cooler, wetter months adjusting less than warm, dry ones.

It's unfortunate that no particular weather station in the nearby rural areas has an unbroken data set for any long interval, they just came and went with various decades covered. So there's nothing absolute that I can use for a comparison. There is a long record at Kingston, Ontario at the east end of the lake, but it's probably too far from Toronto to provide the right kind of corrections, with the chance of anomaly regime differences; also that's a very lake-influenced location and despite being on the lake, Toronto only has limited intrusions of cooler air when winds are southeast to east. Kingston is open to southwest winds (which prevail in these climates) off the lake.
M.T. Cranium is offline  
Thanks from:
03-04-2021, 22:25   #20
M.T. Cranium
Registered User
 
Join Date: Apr 2007
Posts: 12,014
Maybe then to save you some work just specify what time interval you want to have monthly means, and whether or not they should be raw data or adjusted to urban heat island reductions. I could post the file of just that information probably within an hour as it would not require much work here. Presumably if the data are in a dedicated excel file, you could then import said block of data into another file where you have the other data for comparison?

(later developments ...)

I created this file for you and anyone else who wants to use it, only the Celsius monthly means for Toronto are in this one, two sets of data, first the raw data as recorded, and second set, adjusted for urban heat island (so progressively larger amounts subtracted as explained).

This should save you some work. If you do happen to average out my daily C numbers, you'll likely find them similar but occasionally off by 0.1 for the reasons outlined above. I am making up a table of that parameter to compare.
Attached Files
File Type: xlsx Toronto_monthly_means.xlsx (64.1 KB, 1 views)

Last edited by M.T. Cranium; 04-04-2021 at 06:36.
M.T. Cranium is offline  
Thanks from:
Advertisement
06-04-2021, 07:23   #21
Oneiric 3
Registered User
 
Join Date: Jan 2013
Posts: 5,119
Quote:
Originally Posted by M.T. Cranium View Post
Yes those are raw daily means, but the monthly averages derived from them can be found in another part of the spreadsheet, they were averaged by EC and should all be within 0.1 rounded of what you would get from the work on those daily numbers. Also my tables for Toronto run top to bottom without a space for missing 29 Feb (the NYC tables have that feature) so that formulae for some columns would be different than others. Here's the grid references (range) for the monthly averages in the spreadsheet.

They are located in rows 1840 to 2021 to make it easier to figure out to which year the data belong. The columns in play are HW to II. That gives you Jan to Dec and annual means. Unfortunately I did not block the data to convert to one decimal for round numbers, e.g. you'll see April 1842 says 7 which is 7.0.

Let me know if you find this data section and have any questions. The website data are derived from their (EC) averages of monthly mean max and mean min rather than directly from averages of mean daily. It won't make much difference but some of these data would be 0.1 different from what you would get by taking the average of daily means. Once again, my table of daily means is derived from their conversions of the original F data whereas the tables above (the ones that are in F deg) are calculated by me from the entries in the original grids.

If you still wanted to average my data rather than using this table, be aware then that each column runs unbroken from 1 Jan to 31 Dec so that leap years will run from (a) days 1232-1260 for February and (b) days 1261-1291 for March compared to 1260-1290 for non-leap years (incl 1900), etc to end of columns.

My averages at the end of each row for dates have the formulae worked out for that. I suppose I should standardize this work with the NYC tables which I altered for missing leap year dates so that all similar dates of the year after Feb 28 are in the same row. Maybe the next version will have that built in. I'm nervous about throwing off formulae although they should convert along with the shifts.

Also minor point but the -7 and -3 for Jan and Feb 1840 are just rough guesses based on data from Providence RI, there was no weather station at Toronto until 1st of March 1840.

By the way, if you're transcribing these numbers into an excel file, then they also exist in print in the net-weather thread, in the second post of the thread.

You asked if they were raw data and the answer is yes, including those monthly means I mentioned.

For comparison with other data sets, I would recommend applying the urban heat island correction as follows: data 1840 to 1880 is okay as is. Data 1881 to 1890 reduce by 0.1 (to colder values), data 1891 to 1900 reduce 0.2, etc until you reach 1971 to 1980 which would reduce by 1.0 (C deg). After that 1981 to present all reduce by 1.1 C.

If it would save you some work, I could post an excel file of those conversions, that does not yet exist in the excel file but it does exist in print in the net-weather thread, this time on page two after the data on dry spells is tabulated (go to page two of the thread, second post).

If the purpose is to compare with CET or Irish data, I would prefer that these adjusted values would be compared, otherwise there will appear to be a spurious warming trend in the Toronto data over the 20th century into recent decades. If it's to compare with the re-analysis, I don't know what to recommend because I don't know if they factored into their numbers any result of the change in the environment from small city (in 1900) to metropolis (after 1960). By 1900 Toronto was already about a quarter million people with the weather station near the centre of the core, so an urban heat island would already be in place then. It would have grown approximately as suggested by my corrections thereafter. By about 1960 Toronto had surpassed two million population and had spread out a good 20-25 miles beyond its 1900 extent. Changes since then have been relatively minor with most of the growth out beyond the 1960 suburban belt, and tending to cluster rather than run out unbroken, due to greenbelt and river valley intrusions. So I don't think the urban heat island grew very much after about 1950-1960 and the data suggest that also. I used to run my own backyard weather station in a perfect spot for urban-rural comparisons (30 miles west of the downtown station) so I had about ten years of data for comparison, although the location where Toronto downtown is located would be a bit warmer than my location anyway due to lower elevation and closer proximity in winter to Lake Ontario. A sample of some very cold clear night readings would average about 6-8 F deg colder at the MTC-jr location. And that was in a small heat island of a smaller town, really rural settings might be another 4-6 F deg colder again. Most daytime readings were 1-2 deg lower there. However that was complicated by cooling lake breezes that would be just about extinct at my location inland, so some days in March to June in particular would run cooler in the city. It looked to me like that season extended into July more frequently in the 1840s and 1850s before there was any real urban development to weaken the lake breezes (and probably after some of those winters the lake was pretty cold into the middle of summer too).
Thanks for this and the others well detailed info MT. I was eventually going to compare this Toronto data with that of the CET, but my priority for now is to compare it with its model reanalysis equivalent just to see how well the reanalysis (EC 4th Gen) has done in capturing the relatively recent (since 1950) historic temperature trends in that region. I did, a number of years ago, do similar for the 'IMT' series and while general broad scale trends from the EC grib data were more or less similar with the raw data, actual values rarely were, particularly regarding daily and monthly extremes (Max/Min/wind/pressure etc)
Oneiric 3 is offline  
06-04-2021, 07:32   #22
M.T. Cranium
Registered User
 
Join Date: Apr 2007
Posts: 12,014
Okay, so in that case I would suggest comparing the second data table generated (urban heat island adjusted) with the CET, and both with the reanalysis to see what you think of each of them.

I did construct a table of my file means in C and found the expected slight scatter within 0.1 from the official means, with about 80% overlap. However, there were no outliers at all 1991-2010 which made me wonder if they switched their methodology in 1991 (as they would get the same values as me assuming I had no typos from their tables which are the sources of the C data).

On the annual scale there is already a comparison graph available in the excel file for annual means and CET annuals, GL produced a graphic from that. Since it compares raw data it shows the Toronto means catching up to the CET, nowadays there is little difference, back before the city was very large, it was often 1 to 2 C deg colder than CET. The urban adjusted data (on a 5 yr rm basis) follow the CET fairly closely and remain in that range of 1 to 2 deg colder.

The peaks of warmth around 1921 and 1948-53 are similar, as is the warming from about 1987 to 2012. A few minor differences can be seen also, the rise in the 1890s is a bit more prominent for Toronto.

I have not gotten around to comparing months yet, that might reveal relationships which we don't see from just the annual data.
M.T. Cranium is offline  
(2) thanks from:
04-05-2021, 04:39   #23
M.T. Cranium
Registered User
 
Join Date: Apr 2007
Posts: 12,014
I am adding some interesting data to the netweather thread, basically it is a continuous updating of record highs from a set of "starter highs" for 1869-1900, year by year, to put one into the mind set of the person living in those past years, what they considered to be record breaking, and how it all fits together in terms of frequency ... interesting in several ways, including the comparison with UK/Ireland weather which was often similar and sometimes opposite.

This is complete through 1940 and under development for the rest of the period.

From this I have already made a new discovery (new to me anyway) about pulses of warming in the past.

It will take a while to complete the study but I have the preliminary numbers which get adjusted as I check through the data. What that shows is that for the whole period since the late 19th century, there have been regular pulses of warmer weather at intervals of about 15-16 years until the 1970s, then this pulse seemed to accelerate and its period since then has been 11-12 years.

While it sounds like it should have a connection to solar variation, the actual plot of the curves shows that basically there was no strong correlation, the longer period was hitting at various times in solar cycles that peaked between 1905 and 1947, then it began to phase with them after the 1979 cycle, but was totally out of sync during the strong solar cycles of 1947-49, 57-59 and 68-72.

Not sure what to make of this pulse, it seems to have an origin in either the Pacific or the Gulf of Mexico, or both acting together. Those are source regions for warmth impacting the NYC region, the Atlantic is closer but record breaking warmth reaching NYC usually tracks from around Oklahoma and Kansas east-northeast through the Ohio valley. And two times out of three, a warm pulse hitting this region will show up 3-5 days later over western Europe. This is not a guaranteed 1:1 result, some warm peaks in eastern North America never show up further east. The 1936 heat wave for example did not have any later response feature in Europe, it was cool and unsettled for weeks after that.

None of this really tells us much about the human component of climate change. It could range from any conclusion between no connection to major driving factor, but I suspect that the climate of NYC since 1890 looks quite similar to what it would have been on a planet without human advanced civilization, and then if we accept that the greenhouse effect and urban heat islands are two different outcomes of human activity, and take away the urban heat island from the temperature record, that leaves us with a base climate signal that may or may not have been modified by human activity. I suspect it has been modified by some fraction of a degree (the urban heat island is larger). But these pulses may be an entirely natural phenomenon -- they seem to correspond to larger El Nino events.

Despite all the research that has been done, I don't think anyone would be prepared to offer an "advanced" theory of how human activity drives a stronger Pacific oscillation (or how it leads to stronger El Ninos). If anyone thinks otherwise, I'd be interested in hearing the reasoning, and I am quite open minded about all these issues, my main position is that I don't know as much as I would like to know, and you can't make predictions without a knowledge of predictive theory (not on a regular sustainable basis anyway).

Now here's a finding that will surprise some. The ten-year period with the most sustained daily records (the ones that exist as of now) is not that recent, it is 1944 to 1953. The frequency was 4.5 per year (random would be 2.5 over the 152 year record being studied). A secondary peak was 1993-2002 (4.1). A slight tertiary peak (2.9) was 2006-15. The pace in 2019-2020 and first third of 2021 is about equal to random expectation.
M.T. Cranium is offline  
Thanks from:
07-05-2021, 22:07   #24
M.T. Cranium
Registered User
 
Join Date: Apr 2007
Posts: 12,014
That study is getting into the recent past now, have pretty much finished the 1960s and everything before 1960 is done (apart from any editing that may be needed). For NYC, the period 1944 to 1955 was probably just about as warm as the "modern warming" especially if we accept that the urban heat island was only at about 80% of its eventual power to warm overnight temperatures. About 15% of all eventual record highs fall into this twelve year window (which is 8% of the time available). It's interesting now for me because I'm old enough to recall the weather of the 1960s, in fact I was in NYC on New Years eve of 1965 which produced the daily record, we had family there and sometimes visited from our home in Ontario, travel was by bus as I wasn't yet a licensed driver (later in 1966 I was and that was the last of the bus trips).

The summer of 1966 was a scorcher both in NYC and back home in Ontario, very low humidity compared to some of the heat waves we got over time. You'll see reference to a late August early September 1973 heat wave, that one I experienced from many different locations as I was on a road trip with some visiting UK relations, coming back from the western U.S. through Arkansas and Tennessee where it was not only that hot but way more humid than what I was used to seeing, like 82-84 F dewpoints. We drove through Washington DC maybe first or second of September on the way to visit those same NYC relations, and it was about 102 F there. That heat is worse than 117 F experienced at Las Vegas in August 2011 (it's a dry heat and fine as long as you don't touch anything metallic). And it cools off to about 90F by morning (in Las Vegas, almost tolerable for the first hour or so of daylight).

Will be coming to some epic warm spring weather in the late 1970s in the next day or two, and hoping to have this whole historical "deep dive" finished by about Monday or Tuesday with the more recent stuff that I missed out on by moving west in 1995. Looks like that was a very good idea from some of these data.
M.T. Cranium is offline  
Thanks from:
Post Reply

Quick Reply
Message:
Remove Text Formatting
Bold
Italic
Underline

Insert Image
Wrap [QUOTE] tags around selected text
 
Decrease Size
Increase Size
Please sign up or log in to join the discussion

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Share Tweet