Advertisement
If you have a new account but are having problems posting or verifying your account, please email us on [email protected] for help. Thanks :)
Private profiles - please note that profiles marked as private will soon be public. This will facilitate moderation so mods can view users' warning histories. All of your posts across the site will appear on your profile page (including PI, RI). Groups posts will remain private except to users who have access to the same Groups as you. Thread here
Some important site news, please read here. Thanks!

Boards Weather Competition history

Comments

  • Registered Users Posts: 1,752 ✭✭✭ Joe Public


    Agree, it's a great waste of time. We could probably sell the monthly weather according to Con Sensus.

    I was always hoping Ken Ring, AKA Gene Derm, would try his luck at the weather competition but he couldn't be enticed in.

    Another character, don't know if he did the weather comp, was Eoin who got nicknamed Eoin the Moan. He used to get very upset if he didn't get his fair share of snow and hated to hear stories of nearby places getting more than him.


    Thanks for keeping up your interest in the boards forecast competition, its survival depends on a steady hand such as yours.



  • Registered Users Posts: 626 ✭✭✭ Tae laidir


    Thanks for the enlightening history lesson. Fascinating to read those early discussions. And thanks to MT for continuing to put in the brawn and brains.

    Interesting idea of selling Con Sensus data. The knowledge of the many seems to beat that of the single expert. I think it removes biases that we all have, including the experts.

    Covid was responsible for me compiling the summary of 2021. I hope to productively waste some more time compiling summaries for past years. You can never tell how some researcher may find these competitions useful many decades from now. I will post the results here.



  • Registered Users Posts: 13,339 ✭✭✭✭ M.T. Cranium


    In mathematical terms it is fairly difficult for more than a handful of contest entrants to beat Con Sensus especially over the full course of a year. You may not know that I also run a similar contest on an American weather forum, although there we only forecast temperatures. I have one winter snowfall feature and a summer max temperature, otherwise, the forecasters try to predict anomalies at nine U.S. locations. The results there are often about the same, with three or four people ahead of the consensus, but last year everyone finished ahead of Normal. That is where "skill" begins in long-range forecasting, if you don't beat normal then you aren't doing much. Net-weather's temperature contest was not tracking either consensus or normal until about 2017 when I started to do it. The outcome of that is very similar to here. The group consensus would have finished 8th out of about 50 regular entrants last year, the two normals were closer to middle of the pack. Oddly, I did a lot better in the net-weather contest than here in 2021, even though the temperature portions are basically the same question. But then we also have an "EWP" (England and Wales Precip) contest that I added to the long-running CET contest starting with the 2017 year (their year starts in Dec and ends in Nov because they like to focus on the seasonal aspect too). My outcome in the EWP was so-so. Back in 2015 I had good runs in both the UK and Irish contests; that was the only year I threatened the leader here. Would have won in the UK if their 2015 had been annual since I had a top score in December sensing that warm month coming. They had a character on net-weather who had several different names, Craig Evans who always predicted some absurdly high value every month, finished dead last every year, but was famously short of Dec 2015 when he won that month. This was brought up quite regularly for months afterwards. My punt was around 7.5 but it was good enough for second (the month finished at a ridiculous 9.7 which was 1.6 above the previous Dec record high over 356 years at the time).

    There is some group skill involved, I believe. We should all be ahead of NormaL though, if so then our consensus would probably be even harder to beat. However, if you have within the group an equal number of mild and cold "rampers" by nature then it will very likely cancel out and leave the consensus as strong as it would have been without those rampers' forecasts. The place where we tend to fall back towards Normal as a group is when a month just changes completely near the end, if this is the case for PRC and SUN then they are bound to end up closer to normal than many might have predicted looking at the guidance for the first half of the month.

    My observation being involved in so many contests for so long is that a general consensus emerges within a few days, before the contest deadline, based on what the models are showing. It tends to be GFS-biased because the GFS runs out further. All monthly forecasts have to take a blind guess at conditions in the second half of the month. It's likely that many will study those last few panels and try to imagine what happens next. Or they may have analogue ideas in mind, such as "this looks a bit like 2014 to me" and go with that. I also recognize that some predict what they hope will happen, it's like doubling down in blackjack, if you win then you get the weather you wanted (or vice versa). I try to separate out those two things (not with great success obviously) but in my own case what I want to happen is probably whatever I've said in other threads might happen, which can vary from cold to mild or wet to dry. I would rarely go against that initial guess unless the models were just screaming loudly go the other way now.

    Guesses of the unexpected late month reversal are rare and can stick with you for a long time. Nov 2010 is my high water mark and looking back on the signs for that, it was probably as much luck as anything else.

    One other contest that has been running a long time, that I enter but have no part in organizing, is called the Northeast U.S. Snowfall contest; they have a seasonal component and storm-specific contests, one of which is happening tonight, so it's the same story there in terms of group consensus, those have been pretty good for most of the 20 contests (I have entered since around 2009). You see the same names near the top of the results most years too; I had one good run in that and it was in the big snow winter of 2009-2010 which is probably why because I predicted a lot of snow that winter. I've been all over the place since then with my second best effort being last winter but t was only something like 7th out of 25.

    For me, a background motivation is that if anyone does crack the code, they will show up in these contests if they enter, because if you knew with some certainty how things were going to go, imagine the scores you would have, probably in our contest always over 80, and that would end up being 200 points ahead of where our usual winners are. On the net-weather simplified format, your average monthly error would likely drop to 0.4, usually winners there have an average close to 0.8. That U.S. contest that I organize has a monthly maximum possible score of 900 for the nine locations, and we rarely see anyone scoring much over 700 in that; I haven't kept track (it has been running ten years now) but I can't recall a score over 800. But an average error of 0.5 F for the nine locations would get you a score of 810.

    There are probably about half a dozen people in all three contests who are consistently better than normal year after year so they have skill in pattern recognition. The very best in each case are probably about halfway between random and full-skill which in this LRF challenge could perhaps be defined as always within 0.3 C of average temperature and 10% of total precip. If we had working models that could do that (over a grid if not the entire globe) then this would be a settled science. Perfection in long-range forecasting would be hard to achieve even from that close stage, given the complexity, but I think the science would become reliable even at mean errors of 0.5 C and 20 per cent variance on precip. If anyone ever gets to that level they will have "cracked the code" but then there's also a recognition aspect, somebody might do that but not be recognized for it. The contests are probably the only hope they would have to prove to a skeptical world that they had reached that point. Another component would be description of method, to allow duplication. Just a very skilled hunch repeated time after time while impressive would not meet the standard of "scientific discovery" because for that, anyone must be able to replicate the result.


    One thought, for commercial purposes, if you ever wanted to develop the consensus approach, it might be even better if you took a weighted consensus using each forecaster's prior history as the weight. I wonder what weighted consensus would look like in the scoring tables. My guess is that Weighty Con might beat regular Con by 10-20 points a year? Not much but would be that much closer to the desired outcome. Or you could go with a more sophisticated weighted consensus that took into account forecasters' skill levels in each element (as we have been tracking). Some are better at one aspect than another, over the long term. Last year, if you took each category winner's score and added them up, the total would be around a hundred points ahead of our contest winner (Sunflower3) who had one top category finish (minimum temp). The problem would be that these tendencies come and go, so as they say in investing, past performance is no guarantee of future results.



  • Registered Users Posts: 1,752 ✭✭✭ Joe Public


    Interesting idea about "Weighty Con" but the likes of Danno ver1 and Danno ver2 could produce undesirable results.



  • Advertisement
  • Registered Users Posts: 19,654 ✭✭✭✭ Rikand


    Can't believe it was as far back as 2014 that I won. I remember I had a good lead with 2 months to go and instead of going with exactly what I thought, got tactical and started picking forecasts that were in between the two people running closest to me. The win was more important than being right lol



  • Registered Users Posts: 1,412 ✭✭✭ Jpmarn


    It is a pity there is no physical prizes for winning these contests only pride. It is just about looking at the overall weather pattern online and here on the ground. After that it is mainly guesses.



  • Registered Users Posts: 5,925 ✭✭✭ pauldry


    Ya they should give 1 million to anybody who wins 2 years in a row especially if those years are 2017 and 2018.

    Seriously though I don't know what I did different them years though like Rikland I went sensible with my forecasts in November and December as in near normal IIRC.



  • Registered Users Posts: 1,752 ✭✭✭ Joe Public


    As I remember the snow of 1982, all the snow happened in the first evening and night and was accompanied by strong winds. The temperatures remained below freezing for the following 2 weeks.

    https://www.rte.ie/archives/2017/0109/843556-snow-causes-major-disruptions/



  • Advertisement
  • Registered Users Posts: 1,779 ✭✭✭ dacogawa


    I think this is a reunion that has been a long time coming, over a decade and I'm sure you won't have a moment between the weather folk wanting to take you for pints of stout or a trip to many of our splendid golf courses (just not the Trump one).



  • Registered Users Posts: 13,339 ✭✭✭✭ M.T. Cranium


    I would enjoy it, hedging my bets on whether anyone else would, this past two years has probably amplified a pre-existing tendency to live somewhat inside the confines of my own mind. Bet I'm not the only one though, we could probably muddle through. Might need interpreters present.



Advertisement