Advertisement
If you have a new account but are having problems posting or verifying your account, please email us on hello@boards.ie for help. Thanks :)
Hello all! Please ensure that you are posting a new thread or question in the appropriate forum. The Feedback forum is overwhelmed with questions that are having to be moved elsewhere. If you need help to verify your account contact hello@boards.ie

Advances in weather forecasting

  • 21-01-2019 8:55pm
    #1
    Registered Users Posts: 692 ✭✭✭


    Just a quick one.
    Over the years of weather forecasting realistically has our accuracy of forecasting improved? Short term yes I would say a lot.
    Out to day 5 and beyond I would say not a lot for the advances in technology. (Big anticyclonic set ups aside)
    Very much Open to opinions. Are we still missing something or are we not utilising the technology we have??
    Prob won't be a quick one!


Comments

  • Registered Users Posts: 1,475 ✭✭✭Jpmarn


    I find that Met Éireann and the UKMO are the one of the best forecasters in the world. I've been to several foreign places and recently been to Australia, Hong Kong and Singapore. I find the forecast information to be pretty inaccurate. I think they have a good service in America where they can get a lot of precarious weather.


  • Registered Users Posts: 13,494 ✭✭✭✭joujoujou
    Unregistered Users


    Jpmarn wrote: »
    I find that Met Éireann and the UKMO are the one of the best forecasters in the world. [...]

    Honestly, with over 300 rainy days a year here it's not really hard to predict what weather will be. ;)


  • Registered Users Posts: 14,316 ✭✭✭✭M.T. Cranium


    Well your question is really have the computer models improved, the human element in weather forecasting out to about ten days at least is fairly insignificant as every source you might turn to is looking at the same guidance. Sure there's some forecaster skill or experience that comes into play especially when guidance is scattered or marginal for some kind of outcome. That probably hasn't changed, in fact it might have eroded a bit since back in the day forecasters had to analyze their own maps and then draw their own prog charts with some assistance from very rudimentary guidance. As recently as 1980 in a forecast office (I was in one so I can attest to this) the 96h computer model charts were regarded about the same way 10-15 day charts are regarded now, and we would be happy if the 48 and 72h panels proved to be reasonably close to reality. So that part of the technology has improved. The forecasters back then did not slavishly follow those 48h and 72h charts and sometimes managed to improve on them using their experience in similar situations. The idea that half a century ago, people had almost no idea what was going to happen is a bit overblown, weather forecasting was probably better than you imagine even a hundred years ago.

    Now as to the question of can we improve ... one approach is of course to throw much money and resources at "improving computers" to glean even more data and process it through to a finished product. The philosophy of my research is, fine go ahead and try that, it might work, I don't know, but what I do suspect is that theory could also improve forecast models if accurate theory is developed, as to what actually causes weather events. We already know more about climate than weather. We understand in basic terms why the climate of any given location is what it is, but we have no working theory of why the weather events within that climate occur (on the specific dates that they occur, we might have ideas about frequency and even seasonal variations because of drivers or teleconnections). But to take today's cold northwest flow as an example, why did that happen today and not two days back, or this day last year, or 22nd January 1876? (perhaps it did, I didn't look at the archives). And there's nobody out there reading this who can answer that question. A few, including myself, are working on theories to explain such variations. I have published what I already have by way of evidence, and I continue to say that the work is a long way from complete and I hope to get further into this before passing. That's probably a subject for a different thread.

    But what that tells us (and we might not like to admit this) is that meteorology is not a true science, it's a technology. A true science would be based on predictive theories that produce accurate results, like astronomy uses to tell you when the moon is full and even when it will be in eclipse. In fact, they had some of that worked out thousands of years ago. Somehow, with all the intelligent people who have spent their lives doing weather research, we have so far failed to reach the threshhold of true science with even (to give us a fighting chance) a reasonably accurate 30-day forecast model that matches the accuracy of today's 24 hour computer guidance.

    And the obvious reason for that is -- we don't know what causes these weather variations. Now the problem is that many in the field will allow their minds to cloud up with frustrated anger when they read that and go into a "jet stream energy blah blah" sort of "explanation" which I would call metsplaining, a non-explanation dressed up as a "reason" to get the pesky children to go away. But think about this -- if in fact my general approach is correct and the explanation lies in some sort of complicated interference patterns in the atmosphere set off by externally generated gravitational or geomagnetic disturbances, then sooner or later, as with astronomy's advances in understanding orbital dynamics, we would as a community figure this out in detail and get a working model up and running. I've gotten about halfway to the goal of this (I believe anyway) by deriving some parameters for the grid of the interference pattern and some timing aids for low pressure crossing timing lines in that grid. Ireland is near a timing line and I've already discovered that the pressure in the winter months reaches low points about 10-15 mb lower than average near the full and new moon dates. This partially answers one question, why does the ocean respond to lunar tidal forces but not the atmosphere? Well, the main reason why not is that the atmosphere has no coastlines, if we had a global ocean the tides would be equally subtle and hard to detect. But with coastlines, the tidal forces run out of ocean so the water has to return to equilibrium, hence high and low tides. With the atmosphere, the energy is continually rippling around and forming interference patterns. These are harder to model, and because equal numbers of observers are near or distant from timing lines, they form different impressions of how the weather correlates with lunar phase. If you lived between two timing lines (as you would in Berlin, the one place this question was studied in the 19th century) you might be more likely to draw the opposite conclusion, that full and new moons would be times of higher pressure.

    So my suggestion to the community (all along since I started this around 1980) is to fund this kind of research, take it seriously, and get thousands of minds investigating it, not just me and a few other cranks here and there, and it will fall eventually, we'll slowly but surely get better and better results. I don't think there's some magic formula like e equals m c squared in the background waiting to be discovered. This will always be more a technology than a lab-experiment sort of precise science. There are thousands of variables in play after the basic external theory is developed too, but I do feel that this approach would not run out of accuracy after thousands of time intervals because it would remain valid at distant points in the future (or past for verification work). A big spanner in the works would be if the actual parameters of the magnetic field played a role, because those have been slowly changing over time. This would mean that weather data from the past would need to be adjusted as to location to become useful in developing said modelling. The north magnetic pole for example has moved from the central Canadian arctic islands to a point between Alaska and the North Pole in just the past seventy years.

    Well, long answer, but bazlers, this also answers your p.m. that I neglected to answer previously, I think.

    And I should say that this is just my approach, I would not be all that surprised if a totally different philosophical approach proved to be the one we should adopt and that this other approach might deliver the goods. But if I were correct on this, the best you could hope to see would be very gradual incremental improvements in the longer-range portions of existing model runs and extension of those into CFS like products with higher degrees of accuracy. It would not be like in 2019 we had no clue and suddenly in 2023 it all fell together and you could get a 22-day forecast that would be 90% reliable. Probably more like a half century at the very least to get those kinds of improvements.


  • Registered Users Posts: 5,030 ✭✭✭mikeecho


    Well your question is really........


    . . ....like a half century at the very least to get those kinds of improvements.

    That's the grand total of what I read.

    But I'll give you a thanks, purely on the basis of your known knowledge and my respect for you.

    But it's far too late.... And I think it'll take me half a century to understand it.


  • Registered Users Posts: 692 ✭✭✭bazlers


    Well your question is really have the computer models improved, the human element in weather forecasting out to about ten days at least is fairly insignificant as every source you might turn to is looking at the same guidance. Sure there's some forecaster skill or experience that comes into play especially when guidance is scattered or marginal for some kind of outcome. That probably hasn't changed, in fact it might have eroded a bit since back in the day forecasters had to analyze their own maps and then draw their own prog charts with some assistance from very rudimentary guidance. As recently as 1980 in a forecast office (I was in one so I can attest to this) the 96h computer model charts were regarded about the same way 10-15 day charts are regarded now, and we would be happy if the 48 and 72h panels proved to be reasonably close to reality. So that part of the technology has improved. The forecasters back then did not slavishly follow those 48h and 72h charts and sometimes managed to improve on them using their experience in similar situations. The idea that half a century ago, people had almost no idea what was going to happen is a bit overblown, weather forecasting was probably better than you imagine even a hundred years ago.

    Now as to the question of can we improve ... one approach is of course to throw much money and resources at "improving computers" to glean even more data and process it through to a finished product. The philosophy of my research is, fine go ahead and try that, it might work, I don't know, but what I do suspect is that theory could also improve forecast models if accurate theory is developed, as to what actually causes weather events. We already know more about climate than weather. We understand in basic terms why the climate of any given location is what it is, but we have no working theory of why the weather events within that climate occur (on the specific dates that they occur, we might have ideas about frequency and even seasonal variations because of drivers or teleconnections). But to take today's cold northwest flow as an example, why did that happen today and not two days back, or this day last year, or 22nd January 1876? (perhaps it did, I didn't look at the archives). And there's nobody out there reading this who can answer that question. A few, including myself, are working on theories to explain such variations. I have published what I already have by way of evidence, and I continue to say that the work is a long way from complete and I hope to get further into this before passing. That's probably a subject for a different thread.

    But what that tells us (and we might not like to admit this) is that meteorology is not a true science, it's a technology. A true science would be based on predictive theories that produce accurate results, like astronomy uses to tell you when the moon is full and even when it will be in eclipse. In fact, they had some of that worked out thousands of years ago. Somehow, with all the intelligent people who have spent their lives doing weather research, we have so far failed to reach the threshhold of true science with even (to give us a fighting chance) a reasonably accurate 30-day forecast model that matches the accuracy of today's 24 hour computer guidance.

    And the obvious reason for that is -- we don't know what causes these weather variations. Now the problem is that many in the field will allow their minds to cloud up with frustrated anger when they read that and go into a "jet stream energy blah blah" sort of "explanation" which I would call metsplaining, a non-explanation dressed up as a "reason" to get the pesky children to go away. But think about this -- if in fact my general approach is correct and the explanation lies in some sort of complicated interference patterns in the atmosphere set off by externally generated gravitational or geomagnetic disturbances, then sooner or later, as with astronomy's advances in understanding orbital dynamics, we would as a community figure this out in detail and get a working model up and running. I've gotten about halfway to the goal of this (I believe anyway) by deriving some parameters for the grid of the interference pattern and some timing aids for low pressure crossing timing lines in that grid. Ireland is near a timing line and I've already discovered that the pressure in the winter months reaches low points about 10-15 mb lower than average near the full and new moon dates. This partially answers one question, why does the ocean respond to lunar tidal forces but not the atmosphere? Well, the main reason why not is that the atmosphere has no coastlines, if we had a global ocean the tides would be equally subtle and hard to detect. But with coastlines, the tidal forces run out of ocean so the water has to return to equilibrium, hence high and low tides. With the atmosphere, the energy is continually rippling around and forming interference patterns. These are harder to model, and because equal numbers of observers are near or distant from timing lines, they form different impressions of how the weather correlates with lunar phase. If you lived between two timing lines (as you would in Berlin, the one place this question was studied in the 19th century) you might be more likely to draw the opposite conclusion, that full and new moons would be times of higher pressure.

    So my suggestion to the community (all along since I started this around 1980) is to fund this kind of research, take it seriously, and get thousands of minds investigating it, not just me and a few other cranks here and there, and it will fall eventually, we'll slowly but surely get better and better results. I don't think there's some magic formula like e equals m c squared in the background waiting to be discovered. This will always be more a technology than a lab-experiment sort of precise science. There are thousands of variables in play after the basic external theory is developed too, but I do feel that this approach would not run out of accuracy after thousands of time intervals because it would remain valid at distant points in the future (or past for verification work). A big spanner in the works would be if the actual parameters of the magnetic field played a role, because those have been slowly changing over time. This would mean that weather data from the past would need to be adjusted as to location to become useful in developing said modelling. The north magnetic pole for example has moved from the central Canadian arctic islands to a point between Alaska and the North Pole in just the past seventy years.

    Well, long answer, but bazlers, this also answers your p.m. that I neglected to answer previously, I think.

    And I should say that this is just my approach, I would not be all that surprised if a totally different philosophical approach proved to be the one we should adopt and that this other approach might deliver the goods. But if I were correct on this, the best you could hope to see would be very gradual incremental improvements in the longer-range portions of existing model runs and extension of those into CFS like products with higher degrees of accuracy. It would not be like in 2019 we had no clue and suddenly in 2023 it all fell together and you could get a 22-day forecast that would be 90% reliable. Probably more like a half century at the very least to get those kinds of improvements.

    I’m a bit late replying but as always a quiet an educational reply.
    If anyone is interested MTC has a Thread dedicated to his LRF theory on the gravitational effects of our close neighbors.. in Netweather forum. (Simplistically put)

    I really do believe he is on to something and I have requested to the Mods if MTC has the time and energies maybe we could start something here aswell. When the winter is over and things calm down a bit weather wise.

    I do admit I don’t fully understand this theory and probably won’t as I believe “I am of little brain” as the great Pooh the bear would say.
    But I do believe there are plenty here that could contribute if they had the belief in the research and time and energy to build on MTCs great work.

    I have said before I believe MTCs forecasts are usually quiet accurate and he must be on to something... but what I do like, is that he would be the firstr to admit if things went pear shaped and have done and I am sure will again but I guess that’s when you learn the most.

    This is just my view and there are plenty who would have the opposite view or somewhere in between who by debate may hold the key to answering why some forecasts go awry.

    Bazlers.


  • Advertisement
  • Registered Users Posts: 1,785 ✭✭✭piuswal


    A start at answering the question:

    Forecasting the weather requires;

    1.a measurement of the initial conditions and
    2. mathematical model which can then be used to project forward in time the initial conditions


    See
    https://www.metoffice.gov.uk/learning/making-a-forecast
    https://www.metoffice.gov.uk/learning/making-a-forecast/hours-ahead/mesoscale

    A system called numerical weather prediction (NWP) forms the basis of modern weather forecasting.
    The system uses a mathematical model of the atmosphere which has been derived from the laws of physics.
    This model provides a set of equations to solve in order to predict the future weather. These equations are solved by averaging over 'chunks' of the atmosphere (grid boxes) and short periods of time (time steps) to then give us numerical equations which are put into the supercomputer.
    Model resolution
    The size of a chunk is called the 'resolution' of the model, similar to the resolution of an image from a digital camera.
    More than one grid box is needed to represent weather features in the much the same way as a digital image needs more than one pixel to represent something like a face. The number of equations which have to be solved depends on the total number of grid boxes.
    As the equations have to be solved long before the weather happens, the size of grid boxes (given the forecast area) is limited by how quickly they can be solved.
    The current global forecast model has a horizontal resolution of about 40 km over the UK, meaning 160 million equations have to be solved just to step the atmosphere 15 minutes in time. This resolution is very good for information about the general weather conditions over the UK, and the Met Office's current computer power means a 5-day forecast can be produced in a few hours.
    At shorter range (1-2 days) a higher-resolution model (about 12.5 km) is used, because it provides more regional detail.


    https://www.metoffice.gov.uk/learning/making-a-forecast/days-ahead
    https://www.metoffice.gov.uk/learning/making-a-forecast/numerical-models

    Dynamics Research

    Dynamics Research develops and maintains the dynamical core of the Met Office's Unified Model and undertakes research on numerical techniques for use in future cores.


    • About Dynamics Research
    Dynamics Research is responsible for all aspects of the dynamical core of the Met Office Unified Model.
    • Even Newer Dynamics for General atmospheric modelling of the environment (ENDGame)
    ENDGame is the current operational dynamical core for the Unified Model and is based on a semi-implicit semi-Lagrangian discretization of the governing equations.
    • GungHo - a next generation atmospheric dynamical core for weather and climate modelling
    To be able to run effectively on the next generation of supercomputers, future atmospheric dynamical cores will need to scale on hundreds of thousands of processors.
    • Continuous equation sets and approximations.
    The partial differential equations governing atmospheric flow are the subject of this topic.
    • Conservative semi-Lagrangian transport schemes
    SLICE (Semi-Lagrangian Inherently Conserving and Efficient) is a semi-Lagrangian transport scheme that inherently conserves mass.
    • Physics-Dynamics Coupling
    The tendencies due to the physical parametrization schemes need to be coupled to those of the dynamical core.


    Also see;
    https://www.ecmwf.int/en/research/data-assimilation
    https://www.ecmwf.int/en/research/modelling-and-prediction

    Atmospheric physics
    • Atmospheric dynamics
    • Atmospheric physics
    • Atmospheric composition
    • Marine
    • Land
    • Quantifying forecast uncertainty
    • Forecast evaluation


    Atmospheric physics is a vital part of a weather forecast model and is often referred to as the physical parametrization. ECMWF research focuses on how to represent unresolved physical processes in the atmosphere, such as radiation, clouds and subgrid turbulent motions.


    The reference to unresolved processes points to a major issue in limiting forecasts; not all of the phsical processes are resolvedand hence models are far from perfect.

    Another limiting factor is measuring the initial state of the atmosphere; and remembering the fast number of iterative steps that are necessary to produce forecasts, small initial errors increse and incfease. Thtais why one technique currently beibg used is run ensembles of forecasts. The initial conditions are varied slightly , maybe 50 times and the model is then run 50 times and the results are compared. Usually the forecasts will gradually tend to diverge at some time but the longer they stay close together the greater the confidence out to that time.


    All of this of course requires enormous computing power and so therein lies another limitatio;the amount of computing power available.

    There is a vast amount of info on weather forecasting, developmeent and research but the few links Ive included, to UKMO nad ECMWF will provide anyone with a vastsource of material. All it takes is time to read and follow all of the links.



    The attached image covers the processes and interactions that feed into producing our weather and so have to be modelled and as of yet not all of the processes are fully understood or modelled mathematically..


  • Registered Users Posts: 14,316 ✭✭✭✭M.T. Cranium


    Between the two posts of mine and that recent one, you can see what would drive the science forward, namely, recognition of energy cycles that start at a future date. I have total confidence that the science as it exists is as well developed as it can be, it's not a flaw in technique that causes inaccurate model output (the main cause of inaccurate weather forecasting, sometimes the human subjective component lets us down also). It's simply that the models are not given data that approximate future energy cycles of 7 to 12 days in total life span.

    I've never had the opportunity to collaborate with any NWP model managers to see what would happen if they incorporated from time zero some of the concepts I have derived. And I don't have sufficient background in that aspect of the technology to create such a thing myself although I do it without a computer the old-fashioned way. Even that would work fairly well if we knew precisely where to place the energy and what sort of environment it was going to operate in.

    To give a recent example, this low pressure area that moved southeast on 31st of January and ended up in France ... if that was a storm track derived from the research and known to exist two weeks ago, then it could have been forced into the GFS 14-16 day output and perhaps changed it from the unbroken very cold northeasterly it was then showing (if you recall, at one point the 510 dm thickness contour made it to Ireland), to something more closely approximating the end result. It would be unlikely that the same pattern would have sustained that storm track (at that intensity) without backing off on the cold outflow from Scandinavia. So could I have done that? Here's part of the problem ... information overload.

    Once upon a time, I had this idea that if I just kept forecasting continuously (with mental health breaks a few times a year), I would develop enough feel for how the research blended into the forecasting that I would make progress in that regard (fixing up model errors past day five to make it simple). At some point around the mid 1990s I got to a point with the theory that I was able to make some pretty good progress anticipating actual outcomes over North America and so when the internet got going to the extent of having weather forums, I signed on with Netweather hoping to learn a lot more than I knew then about European weather to see if the theory might work as well as I thought it was working in North America. This has never really led to much more than a great deal of specific weather learning on my part (I do know that I have come to understand European weather much better than I did before 2005). I think I need to set aside more time now to review the research model as it exists and as it has performed since about 2010. Sometimes it has led to fairly good forecasts, in a few cases I have had feedback indicating better forecasts than otherwise available. Other times it seems to predict scenarios that the GFS also starts to predict in its longer range then loses in the near term. That intrigues me, two possible reasons come to mind (and there could be others).

    One possible reason for that odd coincidence is that the climate change situation is overwhelming the natural processes which the computer model also picks up on, then has to discard when they fail to reach fruition. The other possible reason is that the grid has changed and that signals now go off at angles to where the research model of mine said they should go, but that still leaves a mystery as to why the GFS which is not using my grid in any way would do the same thing.

    Anyway, bazlers has kindly linked to that netweather discussion thread, and it has been mentioned in other threads on boards in the past too, so people know it's there and although it is rather dense reading I would freely admit, it's not that complex, the problem may be more that it sounds too far-fetched to be possible (even if it worked at 100% accuracy there might remain that problem of sheer acceptance).

    The North American forecasting still goes reasonably well although I have thrown a lot of my mental energy into trying to solve this larger question (have not entirely ignored eastern Asia, South America and Australia over this past two decades, I try to look in at the patterns and compare them to the research model on at least a weekly basis). It is a lot to manage in one's head too, in fact I can't do it, and have to keep referring back to computer files and research notes to keep on top of the details. I remain hopeful that this will work into something at the sort of level that can at least be passed on because there has to be a time limit, fortunately I am blessed with good health and my mental faculties don't seem to be eroding too quickly anyway. I could be at this another ten years possibly. But then possibly not and things are not really in the sort of condition that any survivor of mine could find and transmit everything so that Netweather thread could be the main point of contact for anyone in the future trying to get this theory to a higher level.

    The computer backup for this is nothing very extravagant. I have two or three massive files of daily and monthly data for selected locations and breakdowns of the research index concepts. There is "stuff on paper" too but most of that could be duplicated by anybody who set off on the same search. So that's where things stand. I do suspect that my work is possibly ill-timed in that I started into it just when two bad things were about to happen, carbon loading (the atmosphere seems to be more sluggish to me than what I recall from the 1970s) leading to odd patterns (I am not so much of a believer in runaway warming or other aspects of the orthodox theory, more of a concern that the machine is running out of tune) ... and the second bad thing, a fast motion of the magnetic field combined with its weakening rather rapidly. This may be dragging responses from point A to B and distorting them. Perhaps I am making really good forecasts for some point in the North Atlantic that is supposed to be Britain or Ireland when I use this technique. Some effort has gone into trying to work this out. In North America it seems more that everything has shifted poleward but I do see some indications of timing differences too (the timing lines are very important to getting this sort of approach to work, if they shift away from where you expect them to be, timing will be off, and also the shape of the response will change).

    Bazlers was saying difficult to understand, and I myself don't claim to understand all of what this theory implies, I'm more like a technician trying to get a signal to work properly, perhaps the sheer physics of that is not always top of mind while doing that kind of work.

    So I will try to open this up a bit more and see if together, a lot of interested parties can make more progress than just one solitary enthusiast. The weather in February looks rather boring so far, so perhaps fairly soon.


  • Registered Users Posts: 1,785 ✭✭✭piuswal


    Further development in improving observations;


    http://www.esa.int/spaceinvideos/Videos/2019/02/Flying_under_Aeolus




    Following the launch of Aeolus on 22 August 2018, scientists have been busy fine-tuning and calibrating this latest Earth Explorer satellite. Aeolus carries a revolutionary instrument, which comprises a powerful laser, a large telescope and a very sensitive receiver. It works by emitting short, powerful pulses –50 pulses per second –of ultraviolet light from a laser down into the atmosphere. The instrument then measures the backscattered signals from air molecules, dust particles and water droplets to provide vertical profiles that show the speed of the world’s winds in the lowermost 30 km of the atmosphere. These measurements are needed to improve weather forecasts. As part of the working being done to calibrate this novel mission, scientists have been taking similar measurements from an aircraft carrying an airborne version of Aeolus’instrument. The pilot flies the plane under the satellite as it orbits above so that measurements of wind can be compared.


  • Registered Users Posts: 1,785 ✭✭✭piuswal


    http://www.esa.int/Our_Activities/Observing_the_Earth/Aeolus/Taking_Aeolus_to_the_next_level




    Profiling the world's winds
    11 February 2019
    Since ESA’s Aeolus satellite was launched in August, engineers and scientists have been carefully checking the information that this pioneering mission is delivering on the world’s winds – and now it’s time for the next phase.

    Although our daily weather forecasts are pretty reliable, they still need to be improved further and to do this meteorologists urgently need direct measurements of the wind.

    However, this is no easy task as extraordinary technology is needed to measure the wind from space.

    Nevertheless, ESA’s Aeolus satellite has been designed to do just this. It carries the first instrument of its kind and uses a completely new approach to measuring wind.

    Comprising a powerful laser, a large telescope and a very sensitive receiver, Aeolus’ ground-breaking instrument works by emitting short, powerful pulses of ultraviolet light from a laser to deliver vertical profiles that show the speed of the world’s winds in the lowermost 30 km of the atmosphere.

    Since this is such novel and challenging technology, scientists and engineers have had their work cut out assessing how the satellite is functioning in orbit and checking the quality of the data it is returning.


    Flying under Aeolus
    Access the video
    For example, they have been comparing this new data with modelled data at the European Centre for Medium-Range Weather Forecasts and have already established improvements to the forecast model thanks to the additional data from Aeolus.

    This will have a positive impact on weather forecast accuracy in general.

    ESA’s Aeolus project manager, Anders Elfving, said, “This satellite mission is certainly a challenging one, but I’m very happy to say that we are now formally out of the commissioning phase, which encompasses the first four months of a mission’s life in orbit when we do all the checks and tweaks.

    “We still have some work to do to make sure Aeolus delivers on its promise as we have to improve on the way the data is processed taking into account the peculiarities of its instrument. And, we must remember that this is a completely new type of mission, so we are learning all the time.

    “We also have field campaigns going on all over the world to help with the process of calibration and validation.

    “This means measurements of the wind are being taken from the ground, from balloons and from aircraft to compare with measurements we are getting from space.


    Comparing wind measurements
    “At this stage, the results are expected to be announced in March.”

    One recent field campaign has been carried out in Germany by the German Aerospace Center DLR. This involved flying an aircraft directly under Aeolus’ orbital path and taking more or less simultaneous measurements with an airborne version of the satellite instrument.


  • Registered Users Posts: 1,785 ✭✭✭piuswal


    http://www.esa.int/Our_Activities/Preparing_for_the_Future/Discovery_and_Preparation/EarthCARE_mission_to_improve_weather_forecasts

    12 February 2019
    Earth observation satellite data are crucial for generating accurate weather predictions. They provide a global picture of the current state of the atmosphere, which is used as the starting point for creating a forecast. But our knowledge of that current state is limited, partly because satellites have so far not been able to collect much information about clouds, which play a key role in the weather. The ESA/JAXA future mission EarthCARE will provide this information to improve our predictions about the future weather and climate.

    Energy in the atmosphere is a balance between incoming light from the Sun, which heats Earth, and outgoing thermal radiation, which cools Earth. By reflecting sunlight back out to space, and trapping outgoing infrared light, clouds play a vital role in controlling the temperature of the planet. To a lesser extent, aerosols – small solid particles suspended in the air – do the same. Aerosols also contribute indirectly to Earth's temperature by influencing the life cycle of clouds.

    Clouds and aerosols are important in the radiation budget
    Clouds hugely affect atmospheric heating and cooling
    Even though clouds play a big role in atmospheric heating and cooling, they remain one of the major mysteries in our understanding of how the atmosphere drives the climate system. It is therefore vital for climate research and weather prediction that we improve our understanding and modelling of clouds and their relationship to aerosols and radiation.

    The EarthCARE (Earth Cloud Aerosol and Radiation Explorer) mission is one of the keys to solving the mystery of clouds. Due to be launched in the early 2020s, this mission will provide unrivalled information that scientists will use to study how clouds and aerosols affect the weather and climate.

    EarthCARE will gather this information using two large instruments. One – Atmospheric Lidar (ATLID) – will be used to study aerosols and thin clouds. The other – Cloud Profiling Radar (CPR) – will collect information about thicker clouds and will observe vertical velocities of cloud particles. These two instruments are the most advanced cloud and aerosol profiling instruments ever to be flown in space. The information they gather will help us better understand how the distribution and transport of clouds and aerosols in the atmosphere affects Earth’s temperature.


    The EarthCARE satellite
    The information that EarthCARE provides will greatly improve models of the atmosphere. Such models are important for weather predictions, which work by combining a huge number of observations – mostly from satellites – with a model of the atmosphere. But how can EarthCARE data be effectively combined with other satellite data and used for weather prediction? ESA Discovery and Preparation recently supported the European Centre for Medium-Range Weather Forecasts (ECMWF) to answer this question.

    "No weather prediction model is entirely perfect," explains Marta Janisková, who led the project from ECMWF. "To improve models, forecasts are evaluated against real observations, and the model is constantly improved based on new knowledge, making it more accurate. This process partly explains why weather prediction is much more reliable today than it was 40 years ago. In addition, new observations of clouds improve forecasts by helping to determine the current state of the atmosphere."

    ECMWF investigated how to most effectively incorporate EarthCARE radar and lidar data into their weather prediction models, in preparation for when the data arrives.


    CloudSat radar data
    The investigation was carried out using data from NASA's CloudSat Earth observation satellite and the NASA/CNES CALIPSO environmental satellite, as these are currently the only missions providing space-borne radar and lidar data of clouds and aerosols on a global scale.

    Using data from these two satellites, ECMWF successfully included radar and lidar observations in their model. As well as finding a way to efficiently include this data, the team found that including radar and lidar observations in their model results in better weather forecasts. They saw the biggest improvements in forecasts of temperature, wind and precipitation.

    "One of the main goals of EarthCARE has always been to improve atmospheric models by providing new data on clouds and aerosols," explains Tobias Wehr, ESA's EarthCARE Mission Scientist who oversaw the project. "But improving weather forecasts was beyond the original aim of the mission. Many scientists believed that this task would be too challenging, but through this study, ECMWF has proved that it is possible, opening up a new and exciting application for EarthCARE data."


  • Advertisement
  • Registered Users Posts: 14,316 ✭✭✭✭M.T. Cranium


    All of the above convinces me that our science has failed to understand how every other science got ahead -- THEORY.

    You can have all the observations in the world (literally) but if you don't develop theory, they could be useless in general terms in advancing the science. I know the more observations we have, the better, up to a point, but I don't see a lot of recognition in weather circles (especially among the professionals, I think the enthusiasts get this more because they aren't doing meteorology full time and probably have other scientific interests) that we are going to keep spinning our wheels and failing on the longer range forecast challenges if we don't develop working theories.

    Theory is how every other science got ahead, and they are all ahead of meteorology (I believe anyway) by a considerable margin.

    I just posted this in another thread where it might not be read by that many, but along these lines -- if you could speak to somebody back in the year 1850, and ask them, which of these three things will NOT be the case in the year 2020:

    (a) Mankind will have walked on the Moon.

    (b) People will have devices in their homes that allow them to communicate with anybody, anywhere in the world, and also to do millions of arithmetic calculations in less than a second.

    (c) accurate long-term weather forecasts.

    Well, we know the answer, and I could have added all sorts of other things that would sound ridiculous to the average learned citizen of the mid-19th century, who might easily have figured that accurate weather forecasts might be possible before any of the other far-fetched things. For example, we could generate electricity from breaking down atoms on a large scale (and we would do so more frequently if the politics of it was not holding us back). People would get heart transplants and live healthy lives. You get the idea, all the other sciences have moved rapidly forward and done incredible things. What have we done? I guess the basics of meteorology were not known in 1850 but even back as far as 1900 they had reasonably accurate weather forecasting on a shorter time scale and all we've done is to refine that a bit (to be honest) and make it more like 4-5 days than 1-2 days of reliable service.

    And we won't get much further ahead until the THEORY question is solved to some general state of acceptance with good outcomes coming from predictions based on the theory. The observations are not going to unlock that door. I don't think we need a lot more observations, we know what's going on now, it's what happens to that over ten to thirty days that proves to be almost insoluble. Yet it's sort of in a hazy state of resolution for those of us looking at North American weather patterns which seem to be somewhat easier to diagnose, for whatever reason. I don't claim to be miles ahead of anyone else either, the word THEORY is what I grapple with every day and have done so with a few interruptions for about forty years now.

    I would say our science (which is really little more than a technology without a predictive theory at its foundation) is about where geomorphology was in the mid-19th century when people were still arguing about how the erratic boulders got dropped off at their eventual resting places. People, we are 170 years behind most of the other sciences. Does that not tell you something? And this new generation of so-called climate scientists are not going to advance the science at all, their only paradigm is to relate everything they see to human economic activity, a pursuit that will reveal almost no useful information of any kind. The expression "climate science" is a bit of a joke really. This is the modern equivalent of the search for phlogiston, or the ether.


  • Registered Users Posts: 1,785 ✭✭✭piuswal


    Numerical weather prediction models are computer simulations of the atmosphere.

    They take the analysis as the starting point and evolve the state of the atmosphere forward in time using understanding of physics and fluid dynamics.

    However, the chaotic nature of the atmosphere and incomplete understanding of the processes mean that forecasts become less accurate as the range of the forecast increases.


    The challenge/difficulty is

    1; proper, accurate analysis of the initial state of the atmosphere An image illustrating all of those data pointshas been posted previously

    2; mathematically modelling all of the processes that are involved in the evolution of the atmosphere; not all of the processes are fully understood and even all processes that are understood have not yet been modelled completely

    3; the sheer numbers of time steps in the models mean that any errors in the initial analysis can be magnified very rapidly (ensemble forecastsnmare an attemptnmwith dealing with this issue); hence the need to have as accurate an initial state as possible - just think of the sheer volumeof data involved; covering the whole surface of the globe and up toat least 35km in the atmosphere - anyone care to calculate the volume?

    3a ; all of the variable within the atmosphere - the different types of surfaces, water, snow, vegitation, cities, plains , mountains, valleys etc etc - they have all to be modelled

    4,; the interaction between the oceans and the atmosphere have to be modelled


    Put simply, it is a colossal undertaking and there is a long long way to go but organisations such as ECMWF, the various National Weather Services, many universities etc are continually working on the task WMO is an agency of the UN and brings all of the NMS together It is one of the few ifnot the only global workwhere literally every nation is involved and most if not all of those involved share their information and progress.

    The simple fact is that is a colossal task with so many variables and incomplete models and understanding of how all the pieces fit together and affect each other.

    To say there is not a theory is wrong

    To use examles of progress in other areas is meaningless Different issues, different reasons to work on the issues; the US developed the atomic bomb because of the resources the US Gov put into the task for fear of losing out to the Nazis etc.


  • Registered Users Posts: 8,219 ✭✭✭Gaoth Laidir




  • Registered Users Posts: 14,316 ✭✭✭✭M.T. Cranium


    I would have to have a conversation with piuswal about what he thinks a theory is, since what I mean is that (self-evidently) there is no working theory that permits accurate prediction beyond the rather limited range of accuracy of numerical weather prediction. Of course there's theory that went into all that, and I know that (how could I not?).

    But that's like saying that the pre-Newtonian understanding of cosmology was a working theory that didn't need gravitation to make us fully aware of how orbital dynamics might work. We are at that early theoretical stage, let's put it that way. Surely nobody is going to try to defend the proposition that we have anywhere a working theory of atmospheric variations? What's the weather map going to look like on January 1st, 2020 then? Your working theory should tell us. I know where the Moon is going to be, or at least I could find out in an instant from my astronomical tables. I know where Jupiter and Saturn are going to be. Where's the Icelandic low going to be?

    And why?

    (a purist will say there is a slight tolerance factor in answers to my astronomical assertions, the Moon might be a second or two late getting to its transit point or whatever -- fine -- but the basic fact is that we don't have a theoretical basis to say what a weather map on a distant date will look like and can only rely on climatology -- averages over time -- which is like saying the Moon will be about 1.2 light seconds from earth somewhere not that far from its orbital plane.)

    You see, this is the whole problem, even people who understand a lot about meteorology have no concept of incompleteness of this so-called science which is technically not a science, it just looks like a science. It will be a science when or if we get much further down the road of understanding how to predict beyond the brief capability of our machines, and when we know the reasons why we expect x or y to happen, just as we have a rational reason to say that the next new moon is on 6th of March. That's not done by gathering thousands of data points, it is done by solving equations based on theory.

    Final point, this whole idea that getting a lot more data will help -- I can't rule it out, it depends on what energy trends are revealed by that additional data. I can't say how long duration those additional clues might prove to be. But my suspicion is that we already have enough data to do all we can under the current theoretical limitations. If we had better theories, we could model weather events from those theories instead of trying to make data collected today reveal weather maps for 16 days from now. The 16-day GFS demonstrates both the potential and limitation of that assumption, sometimes if all the trends revealed are going to persist that long, it will look fairly accurate. Other times it will not (remember the fiascos of January 24-31 this past winter?). A better theory would be able to take the first few days, assess developing errors from the current technology, and keep making course corrections. This is how they get space probes to perform so well, the initial equations of motion are constantly updated from mission control. If they sent a space probe to Pluto or Ultima Thule and hoped it would get there on the initial trajectory commands alone, they would usually fail by a wide margin. This is where we are at, failing by a wide margin. Surely you've noticed?


Advertisement