Forecasting then, and now -- you might be surprised

M.T. Cranium

This is going to be a rambling discourse about forecasting the weather back in the day, as compared to nowadays. It was quite a different world as recently as the mid to late 1980s.

My own experience in forecasting goes back to foundations that are probably similar to many of you on this weather forum. As a high school student, I was marginally interested in the weather but more into astronomy. Despite that, my parents decided to provide me with weather instruments (the kind you have to read manually) at Christmas of 1963 and so I started a daily record of observations that New Years which I kept up for five or six years. When I went off to university and wasn't around the weather station much, a younger friend took on the job and I continued to observe on weekends and in the summer holidays. This was all in a town just west of Toronto where as you can imagine the weather is quite variable and occasionally rather dramatic. Then when I moved north after graduating and lived in what Canadians call "central Ontario" (not that central) I was in a lake effect snow belt and kept up the observations through several winters of rather extreme weather in the 1970s.

However, I never intended to make a career in this field. My education was both math-science and geography oriented and naturally this gave me some courses in climatology, whether or not I graduated as a climate scientist is a matter of opinion rather than fact, but I didn't bother with graduate school, thinking instead that I might want to teach high school (math and or geography). That career was cut short by the untimely death of my mother, herself a teacher, during the final stages of the training year, which basically gave me somewhat more flexibility than I was perhaps expecting, and a chance to travel more (and work less) for a year or two. Eventually, I landed on my feet and determined that maybe I should pursue my interest in the weather. You might have pictured that I was like the most avid 20-something weather geek squared, but to be honest I had my mind more on other things such as playing soccer and well let's not go any further down that road.

In the period 1976 to 1980 then I worked for various private weather companies, first in Toronto and then at Accu-weather where I was hired to help them develop Canadian clients. During that stint, I met Joe Bastardi who is pretty close to my age, and a few other people you might have seen mentioned in their press releases. What I thought might be interesting is how weather forecasting was done in the 1970s and early 1980s, as well as some brief reflections on what it was like even earlier and also since then.

It might be worth exploring very briefly why I was forecasting at all, as I was hired on initially to do technician duties. The Toronto company had a rapid increase in business but couldn't find enough capable forecasters to fill out a 24/7 roster. They had one "senior man" and a couple of part timers on loan from the nearby government offices, but needed more than that. Since I was interested in forecasting (although not really thinking much about being a forecaster) I got sucked in and the rest is history.

In the mid-1970s there were basically two weather models available for forecasting. The better of them was known as the LFM or "Limited Fine Mesh" and it cranked out forecasts for the first 48 hours as well as some experimental maps for 72 and I think eventually 96 hours. Nobody took those overly seriously. All of the weather guidance used to arrive in a weather office on a system called "Alden FAX" which involved some rather stinky chemicals, a continuous feed of paper soaked in those chemicals and newly printed maps that you had to hang up to dry on a sort of clothesline before it was safe to read them. The schedule was pretty much what we're used to, but there were no 06z and 18z forecast maps, just the 00z and 12z runs. There was no European model that we knew about anyway.

Our data arrived on teletype, and on the same schedule that you can see current data nowadays on your computer. There was some limited computer to computer transmittal of data too. Desk-top computers were very new, I can still remember in my university days going into a computer lab where there was a huge machine with vacuum tubes and fortran cards for programming. Bill Gates was probably hitch-hiking around Europe about the time I was working in these weather offices. But while some weather maps came in on the dreaded Alden FAX and could be studied, the practice was to plot and draw up (analyze) your own maps, then you would look at the guidance and draw up your own progs. Many of the concepts that are well known to thousands of weather enthusiasts, not to mention professional forecasters, nowadays were completely unknown back then, but I have to say that the accuracy of 24 and 48 hour forecasts (and maps) was not vastly different.

In fact, I have the feeling that being that hands-on made you more sensitive to nuances that you don't get by just looking at maps and charts, and it could lead to some pretty good forecasts. The accuracy often degraded faster than what we're used to seeing nowadays. It was considered a coup to get to the end of a long weekend without totally losing the plot, so to speak. Sometimes the guidance was horrendously inaccurate very early into a storm cycle, not often, but the higher resolution models we have now would not often do what the 1970s version models did about once a month, which would be to take a storm and track it way off its actual course. I can remember one low in April 1979 during a rather cold spring that was supposed to move due east at about 45-46 deg N and ended up going southeast into Tennessee and North Carolina. You can imagine the havoc that caused in short-range forecasts.

One of my best memories is the "Superstorm of January 26, 1978" which occurred on a day that I was working a shift. We knew the night before that something big was coming, but the models were a bit off on the details. As I got into work, I noted that the air conditioning unit from the roof of the office was now in my parking space thanks to the strong wind gusts. And as I drew up the map, I could not quite believe what I was seeing (we were very lucky to get the observations and to keep the power on). London Ontario had something like a south wind at 60 mph and snow from Lake Erie, and it was about -8 C, while in Toronto it was 4 degrees with rain and an east wind. And the pressure at a station on Lake Huron was 955 mbs. I still have that map around my home office here, quite a sight. I also had to draw up the blizzard off the east coast that followed in about two weeks.

So the tools have been described, but how about process? At first I was working as part of a two or three person team, or sometimes alone. So we would get around a table and discuss what was likely to happen in the forecasts of interest, as there were so few models it was not much of a debate, and most of the time it was all about interpretation issues. At Accu-weather, where you had a much larger group (30-60 forecasters all in one rather small room in what I believe was a converted church) the process was one of managed discussion. A team leader would get people to submit ideas on paper mostly, such as drawing up snowfall maps when there was a regional snowstorm. Then all of them would be pinned up and people would walk around like folks at an art gallery for a while, and have a conversation (an argument, whatever). Consensus was often something like a weighted average, the weight being experience rather than force of argument (sorry Joe, I had to say that).

My big coup in that period was to draw an 18" snowfall max on a map of the big Leap Year Day blizzard of 1980 and have it verify, although not necessarily accepted prior to the fact. After that some of the bigwigs took a bit more note of my eccentricities in precip forecasting which got a severe workout because in the spring of 1980 there was one closed low after another over New York, New Jersey and Pennsylvania where most of the clients were. We had one day with about five inches of rain in northern New Jersey. They were very meticulous about warning clients who needed to know flood potential about such matters. I think it was done as well back then as it has been done since.

A lot of other things in the weather field were quite primitive back then. Television graphics were at an early stage, most of the presenters drew maps either on paper (while you watched) or with chalk, some got adept at drawing numbers in mirror image so it would look right on the far side of a glass screen. Really, not making this up. Discussion of the coming weather was pretty basic and the forecasts were not all that detailed. Warnings were not as prevalent and as the audiences were often more limited in geographical spread, a big city TV station rarely said much about weather in the nearby rural areas or snow belts. There was much less awareness of lake effect snow storms and I did some pioneer research into patterns and forecasting (which some knob at the government agency stole and published as if he had done the work).

My interest in forecasting quickly morphed to global climate and long-range forecasting while at Accu-weather. Before that, I don't think I ever tried to make a monthly forecast. There were very few techniques available back in those days, people seemed to go mostly on pattern recognition and instinct. Some of the results were remarkably good considering that fact but it was the time of amplified patterns and big anomalies so perhaps it was easier than some recent years. By the middle to end of 1980 I had decided to go into that sort of work and I was able to sell long-range forecasts on a mail subscription basis. Publicity was always the most difficult aspect, I was able to retain about 80% of paying clients from year to year but how to reach thousands of them? Also mailing costs got rather prohibitive after a while. I had other work and let this "peter out" in the late 1980s. We could see that something like the internet was coming but didn't know exactly what shape it would take. Meanwhile, more for research than urgent work, I needed weather data and the only way to get it was either walk into an airport weather office and take the expired data printouts, or buy it over a primitive phone dial-up transfer. So I ended up spending many thousands of bucks that I had earned in my day job for weather data which was then absolutely free by 1996 or thereabouts in the very early stages of the public internet.

I have a feeling this post is already too long for boards but I will post it and finish up with a second post that should follow in a few minutes. If you feel moved to post a comment, could you hold off until you see a second post? Thanks.

M.T. Cranium

Just briefly, how about forecasting even further back? We've all read stories about how they predicted the landing weather for D-Day, computers that took almost a full daily cycle to generate output, and other tales. From what I understand, weather offices operated about the same way we did in the 1970s except that before 1960 there were no computer models or satellite images, and only primitive radar. In fact, even in 1979 we had no radar displays where I worked, just numerical data that I had to plot on a map (such as this, 320 50nm east of YXU which meant a top of 32,000' located 50 nautical miles east of London airport). That was better than nothing but we had to rely mostly on satellite imagery and reports for our forecasting, the radar plots just added some details.

Going back about as far as 1900 and perhaps even 1880, the procedures were broadly similar if lacking in detail, you received observations over a teletype system and drew up maps. There was some understanding of the motion of weather systems. Without satellite imagery, major surprises were fairly common. The 1900 Galveston hurricane was only predicted at the last moment after ship reports failed to come in for several days as the storm left Cuban waters and moved through the Gulf of Mexico. And it was only towards landfall that there was much indication of its power. I understand that the 1925 super-tornado outbreak was relatively well predicted in advance, but that forecasters were required not to mention tornadoes specifically, a fact which only changed after the 1947 Woodward OK disaster. The predictions of the 1938 Long Island Express hurricane were rather basic and largely inadequate, but things like the 1936 heat wave were probably quite easy to predict from the information then available.

Keep in mind, the existence of the jet stream was not really understood until around WW-II and upper air maps were not available until about 1941-42.

Some of you may have a lot of details to share about long-ago forecasting methods and abilities in Europe, I don't know very much about that side of the historical record.

Briefly then, my thoughts about how forecasting has changed with the internet and better computer models -- the main improvements are more accurate three to five day outlooks, some advance in longer range forecasts, and a better climate of public understanding of basic meteorology so that forecasters can use various media and give out more detailed outlooks with reference to rather complex information, and find that it meets with an interested and capable set of users (or critics as we sometimes call them).

As you can imagine, the notion of a lone wolf operator such as myself making any kind of forecasts half the world away in real time, and having anyone take that at all seriously, would be entirely impossible before about 1998 to 2000. That might be a good thing. But it is what it is.

What might we have done in the 1970s with today's technology? Quite a few things would be different, I suspect. The entire Great Lakes region would have been on high alert for the Superstorm. As it happened, there was a certain amount of anticipation according to accounts from folks in Ohio and Michigan on an American weather forum, but I don't recall anyone in Ontario being particularly on high alert for it, we were expecting strong winds and heavy snow but back in those days, people tended to react to these things much differently. People would go out on the highways and take forever to get from point A to B (sort of like this thread) with the stoic attitude that there was no better way. Highway clearance was pretty much an after-the-storm event and so there was usually a lot longer period of clearing stranded vehicles out of the way first. Then we had a long interval of rather mild and open winters in Ontario so the more modern philosophies of winter storm management are rather specific to the past two decades.

What will change in the future?

My belief is that forecast models will improve only very marginally if at all, until some day when a working theory of weather variations is developed. I don't mean vague theories like ENSO and PDO type teleconnections, but the sort of research that I am doing where you attempt to model variations from known signals. The ones that I have chosen happen to be external but that isn't necessarily the only set you could choose -- there might be statistical modelling advances (this is already somewhat incorporated into the global models now) or even undiscovered internal signals (geomagnetic pulses not from space or high upper atmosphere, oceanic signals that can be predicted more precisely, etc). If that happens then "Fantasy Island" may extend a lot further out (to some extent it already does in some research paradigms) and what we now call FI might start to resemble today's 3-6 day time scale.

As you may have noted yourself, the 7-10 day panels of the ECM and 11-16 day panels of the GFS are not entirely random nonsense (all the time, that is). The computer models are capable of solving some of the unexplained non-random variance of the atmosphere out to about 15-20 days. Beyond that, I would submit that progress beyond random or persistence standards might be hard to establish. Anecdotal evidence may be offered, but unless we know what causes the highs and lows to form and move, there is little chance of very accurate super-long-range forecasting. I've discussed what I think may be the general paradigm to explore in other times and places. Linking that to operational forecast models is something that only a few highly trained workers could attempt to do. But anyone could draw up approximate maps of what they thought their research factors might produce. I am so busy with a million other things that I have not gone as far into this area of research and work as I would like. It may be that I should give up the interesting and rewarding daily forecasting and other things I do, and get back to how I was working in the rather tranquil days between the publication stage of the 1980s and the on-line work that I started around 2004. Back then, I was working mainly for my own amusement and creating forecasts that were black-box in nature and sometimes surprisingly accurate. It was that finding that led me to want to get involved in European forecasting, I wanted to see if what I was doing in North America to some degree of success could be done globally and this seemed like the best "next step."

Well, I'm running out of years and kind of going around in circles between expanding the research to the point where it is now so complicated that I have to give myself refresher courses on my own files. It would have been a great thing if somewhere along the road I had linked up with a funding agency and acquired some paid helpers, maybe offloaded detailed parallel studies on a hundred motivated persons, and seen this system advanced more. Is it like a northern hemisphere Ken Ring, you might wonder? Well maybe, but my theory is not as fixated on the moon, it draws on a larger set of external signals and the moon is a modulator of other sets of signals that would exist without it. My fear is that I am going to kick the proverbial bucket at some point before I get this all "in order" and/or that nobody will want to continue the work. I have a few quasi-interested younger contacts but I can't really say that anyone passionately believes in the system or wants to take it on.

So because of that, I am going to think about new directions in 2015. At the same time as I face all those challenges, Canadian political activists are fighting some complex "lawfare" battles and I have become a mid-level target of what we feel is a large-scale assault on our security and citizenship. The government is neither behind them or behind us at this point. So I have been wasting a lot of time in court fighting for internet freedom of speech when it was already the case that I was spending too little time on basic research. Yet the forecasting makes the research program better. This is not a foreshadowing of some resignation announcement. I intend to go on doing what I have been doing, but perhaps with a much more determined "put house in order" approach to my research.

And then I shall die, boots on, no doubt. So there is something to look forward to, anyway.

heffoo

Made for fascinating reading peter

highdef

What a fantastic read and quite an insight into your background. What an interesting life you have been living.....and still are, so it would appear and it is important that you get your priorities in order! Rarely a day goes by that I do not read one of your daily forecasts and it would be strange for those forecasts to no longer exist. In fact, your forecast is normally the first thing I look at in the morning when I open up the internet, before looking at any other forums or charts or models. You are my number 1 de-facto forecast above everything else.
There is one thing that would great to see though and this thing would probably help many people (including me) connect even stronger with you and that would be a photograph of you. Perhaps you would like to remain the elusive MT/Peter and if so, stay with it but if you didn't mind, I'm sure a lot of people would love to know that you really do have a face!

Bejubby

highdef wrote: »

What a fantastic read and quite an insight into your background. What an interesting life you have been living.....and still are, so it would appear and it is important that you get your priorities in order! Rarely a day goes by that I do not read one of your daily forecasts and it would be strange for those forecasts to no longer exist. In fact, your forecast is normally the first thing I look at in the morning when I open up the internet, before looking at any other forums or charts or models. You are my number 1 de-facto forecast above everything else.
There is one thing that would great to see though and this thing would probably help many people (including me) connect even stronger with you and that would be a photograph of you. Perhaps you would like to remain the elusive MT/Peter and if so, stay with it but if you didn't mind, I'm sure a lot of people would love to know that you really do have a face!

I can connect with a lot of what you said there about the only and first weather report to be read each morning.

Thanks Peter.

mirrorwall14

What a fascinating read this morning, thanks!

Forever21

What a fascinating read. Once again I read your daily post first thing every morning before I even read the news online. I"m sure like many your part of our daily lives . Hopefully one day you'll visit Ireland & all boardies can meet you at a central location.i know nothing about reading weather charts but I love your posts & I'm learning more about the weather. What an interesting life you have & hopefully you won't be hanging up your boots for many a year yet.

highdef

I think M.T. could be mobbed if he decided to take part in a boards meet up in Ireland. I just did a quick check and I would be quite confident in saying that M.T. Cranium is very likely the most thanked user on Boards.ie with over 100,000 thanks to date! :eek:

M.T. Cranium

Well it is a mutual thing about looking forward to meeting up with people, and I would hope it might take place in a way that stresses the concept that we are all meeting each other as equals if you see where I am going with that. Yeah I have had an interesting ride but I imagine the same is true of all of us. There is no great desire on my part to make a "state visit" sort of an occasion out of a visit to Ireland and I really hope it would turn into something rather general for the weather forum (and IWO, more or less a common pool I think) community. There are other notables that people might wish to meet too, not overlooking that fact, and I truly have no idea whether any of you happen to know one another or not in real life, would imagine a few have met along the way possibly at those Met evenings or such.

My other passion is golf and so any kind of visit would have to include a few rounds of golf, perhaps one could be a sort of boards/weather golf event although I am guessing that 90% of weather forum regulars are not golfers. Still you could do the social part as a follow-up to the golf event.

The other complication is that my wife would likely be along (she has never seen the land, just Britain) and she's not greatly into the weather, so there again, I would have to be realistic about what I planned to do (maybe a separate visit, but we are not wealthy folk who could make multiple trips to Ireland).

As to what I look like, that was actually handled a while back with a photo, just a fairly average looking older guy (turned 65 last June) with a beard and a slightly deranged have fought in wars sort of appearance, as with all weather forecasters. I don't look even remotely like Doctor Evil. You could try friending me on facebook if I feel like sharing, not sure how people do that but I use the other meatworld name there (initials RS not POD). Please don't screen capture any facebook photos as they may show third parties or innocents unrelated to crimes such as weather forecasting.

Bejubby

Mount Juliet for the round of golf,there one thing sorted anyway.
Any other golfers in the weather forum?

genuine leather

Great posts MT, lovely to know something of the man behind the daily forecasts. Your interesting journey and the history of forecasting, thanks.
I have spent, just two years using your forecasts daily, great job.

I wish you and yours a Very Happy Christmas.