With the introduction of the <Attributes/> element in version 0.3.4, and the extended z attribute in v0.3.5, it is now possible to express requests in a more compact manner, thus increasing speed and efficiency while reducing bandwidth.

An Example

Let us consider the following request, which retrieves wind, temperature, and magnetic declination data at four locations, on two different altitudes for two of them:

Request

(1473 bytes indented, 1376 bytes unindented)

<?xml version="1.0"?>
<nwx version="0.3.5">
  <Request>
    <Wind id="0" p="47.0819,-0.8772" z="10" u="F" e="2010-04-23 14:00:00"/>
    <Geomag id="0" p="47.0819,-0.8772" e="2010-04-23 14:00:00" model="IGRF-11"/>
    <temp id="0" p="47.0819,-0.8772" z="10" u="F" e="2010-04-23 14:00:00"/>
    <Wind id="0" p="47.0819,-0.8772" z="230" u="F" e="2010-04-23 14:00:00"/>
    <Geomag id="0" p="47.0819,-0.8772" e="2010-04-23 14:00:00" model="IGRF-11"/>
    <temp id="0" p="47.0819,-0.8772" z="230" u="F" e="2010-04-23 14:00:00"/>
    <Wind id="1" p="47.5368,-0.8518" z="230" u="F" e="2010-04-23 14:00:00"/>
    <Geomag id="1" p="47.5368,-0.8518" e="2010-04-23 14:00:00" model="IGRF-11"/>
    <temp id="1" p="47.5368,-0.8518" z="230" u="F" e="2010-04-23 14:00:00"/>
    <Wind id="2" p="47.8061,0.2738" z="230" u="F" e="2010-04-23 14:00:00"/>
    <Geomag id="2" p="47.8061,0.2738" e="2010-04-23 14:00:00" model="IGRF-11"/>
    <temp id="2" p="47.8061,0.2738" z="230" u="F" e="2010-04-23 14:00:00"/>
    <Wind id="3" p="48.9695,2.4414" z="230" u="F" e="2010-04-23 14:00:00"/>
    <Geomag id="3" p="48.9695,2.4414" e="2010-04-23 14:00:00" model="IGRF-11"/>
    <temp id="3" p="48.9695,2.4414" z="230" u="F" e="2010-04-23 14:00:00"/>
    <Wind id="3" p="48.9695,2.4414" z="10" u="F" e="2010-04-23 14:00:00"/>
    <Geomag id="3" p="48.9695,2.4414" e="2010-04-23 14:00:00" model="IGRF-11"/>
    <temp id="3" p="48.9695,2.4414" z="10" u="F" e="2010-04-23 14:00:00"/>
  </Request>
</nwx>

Response

(2652 bytes indented, 1993 bytes unindented)

<?xml version="1.0" encoding="UTF-8"?>
<nwx version="0.3.5">                 
  <Response expires="2010-04-21 12:20:00">
    <Wind id="0" p="47.0819,-0.8772" z="10" u="F" e="2010-04-23 14:00:00">
      <dir>66</dir>                                                 
      <speed>9</speed>                                              
    </Wind>                                                               
    <Geomag id="0" p="47.0819,-0.8772" e="2010-04-23 14:00:00" model="IGRF-11">
      <decl>-1.7506</decl>                                               
      <ddecl>0.0354</ddecl>                                              
    </Geomag>                                                                  
    <temp id="0" p="47.0819,-0.8772" z="10" u="F" e="2010-04-23 14:00:00">15.9</temp>
    <Wind id="0" p="47.0819,-0.8772" z="230" u="F" e="2010-04-23 14:00:00">
      <dir>339</dir>
      <speed>1</speed>
    </Wind>
    <Geomag id="0" p="47.0819,-0.8772" e="2010-04-23 14:00:00" model="IGRF-11">
      <decl>-1.7506</decl>
      <ddecl>0.0354</ddecl>
    </Geomag>
    <temp id="0" p="47.0819,-0.8772" z="230" u="F" e="2010-04-23 14:00:00">-29.5</temp>
    <Wind id="1" p="47.5368,-0.8518" z="230" u="F" e="2010-04-23 14:00:00">
      <dir>299</dir>
      <speed>4</speed>
    </Wind>
    <Geomag id="1" p="47.5368,-0.8518" e="2010-04-23 14:00:00" model="IGRF-11">
      <decl>-1.8492</decl>
      <ddecl>0.0355</ddecl>
    </Geomag>
    <temp id="1" p="47.5368,-0.8518" z="230" u="F" e="2010-04-23 14:00:00">-29.7</temp>
    <Wind id="2" p="47.8061,0.2738" z="230" u="F" e="2010-04-23 14:00:00">
      <dir>285</dir>
      <speed>8</speed>
    </Wind>
    <Geomag id="2" p="47.8061,0.2738" e="2010-04-23 14:00:00" model="IGRF-11">
      <decl>-1.5492</decl>
      <ddecl>0.037</ddecl>
    </Geomag>
    <temp id="2" p="47.8061,0.2738" z="230" u="F" e="2010-04-23 14:00:00">-29.8</temp>
    <Wind id="3" p="48.9695,2.4414" z="230" u="F" e="2010-04-23 14:00:00">
      <dir>290</dir>
      <speed>18</speed>
    </Wind>
    <Geomag id="3" p="48.9695,2.4414" e="2010-04-23 14:00:00" model="IGRF-11">
      <decl>-1.1824</decl>
      <ddecl>0.04</ddecl>
    </Geomag>
    <temp id="3" p="48.9695,2.4414" z="230" u="F" e="2010-04-23 14:00:00">-30.4</temp>
    <Wind id="3" p="48.9695,2.4414" z="10" u="F" e="2010-04-23 14:00:00">
      <dir>63</dir>
      <speed>8</speed>
    </Wind>
    <Geomag id="3" p="48.9695,2.4414" e="2010-04-23 14:00:00" model="IGRF-11">
      <decl>-1.1824</decl>
      <ddecl>0.04</ddecl>
    </Geomag>
    <temp id="3" p="48.9695,2.4414" z="10" u="F" e="2010-04-23 14:00:00">13.3</temp>
  </Response>
</nwx>

It can be immediately seen that there is a lot of repetition in the above, which is what <Attributes/> attempts to address.
With its help we can express the same request in this more compact form:

Request

(677 bytes indented, 496 unindented)

<?xml version="1.0"?>
<nwx version="0.3.5">
  <Request>
    <Attributes u="F" e="2010-04-23 14:00:00">
      <Attributes p="47.0819,-0.8772" z="10 230">
        <Wind/>
        <Geomag model="IGRF-11"/>
        <temp/>
      </Attributes>
      <Attributes p="47.5368,-0.8518" z="230">
        <Wind/>
        <Geomag model="IGRF-11"/>
        <temp/>
      </Attributes>
      <Attributes p="47.8061,0.2738" z="230">
        <Wind/>
        <Geomag model="IGRF-11"/>
        <temp/>
      </Attributes>
      <Attributes p="48.9695,2.4414" z="10 230">
        <Wind/>
        <Geomag model="IGRF-11"/>
        <temp/>
      </Attributes>
    </Attributes>
  </Request>
</nwx>

Response

(1393 bytes indented, 947 unindented)

<?xml version="1.0" encoding="UTF-8"?>
<nwx version="0.3.5">                 
  <Response expires="2010-04-21 12:20:00">
    <Attributes u="F" e="2010-04-23 14:00:00">
      <Attributes p="47.0819,-0.8772" z="10 230">
        <Wind>
          <dir>66 339</dir>
          <speed>9 1</speed>
        </Wind>
        <Geomag model="IGRF-11">
          <decl>-1.7506</decl>
          <ddecl>0.0354</ddecl>
        </Geomag>
        <temp>15.9 -29.5</temp>
      </Attributes>
      <Attributes p="47.5368,-0.8518" z="230">
        <Wind>
          <dir>299</dir>
          <speed>4</speed>
        </Wind>
        <Geomag model="IGRF-11">
          <decl>-1.8492</decl>
          <ddecl>0.0355</ddecl>
        </Geomag>
        <temp>-29.7</temp>
      </Attributes>
      <Attributes p="47.8061,0.2738" z="230">
        <Wind>
          <dir>285</dir>
          <speed>8</speed>
        </Wind>
        <Geomag model="IGRF-11">
          <decl>-1.5492</decl>
          <ddecl>0.037</ddecl>
        </Geomag>
        <temp>-29.8</temp>
      </Attributes>
      <Attributes p="48.9695,2.4414" z="10 230">
        <Wind>
          <dir>63 290</dir>
          <speed>8 18</speed>
        </Wind>
        <Geomag model="IGRF-11">
          <decl>-1.1824</decl>
          <ddecl>0.04</ddecl>
        </Geomag>
        <temp>13.3 -30.4</temp>
      </Attributes>
    </Attributes>
  </Response>
</nwx>

With help from the <Attributes/> element, and taking advantage of the extended z attribute to squeeze two <Wind/> calls into one we were able to make the request 54% smaller, and obtain a reduction of 47% in the size of the response. Note also how we have used two nested <Attributes/> tags to avoid duplicating the altitude units and epoch information.

But we could take this example even further by using the extended p attribute as well, if we accept to get some extra information which we don't need (the second and third points' FL10 data) in the interests of bandwidth efficiency:

Request

(278 bytes indented, 240 unindented)

<?xml version="1.0"?>
<nwx version="0.3.5">
  <Request>
    <Attributes p="47.0819,-0.8772 47.5368,-0.8518 47.8061,0.2738 48.9695,2.4414" z="10 230" u="F" e="2010-04-23 14:00:00">
      <Wind/>
      <Geomag model="IGRF-11"/>
      <temp/>
    </Attributes>
  </Request>
</nwx>

Response

(592 bytes indented, 504 unindented)

<?xml version="1.0" encoding="UTF-8"?>
<nwx version="0.3.5">
  <Response expires="2010-04-21 12:20:00">
    <Attributes p="47.0819,-0.8772 47.5368,-0.8518 47.8061,0.2738 48.9695,2.4414" z="10 230" u="F" e="2010-04-23 14:00:00">
      <Wind>
        <dir>66 74 72 63 339 299 285 290</dir>
        <speed>9 8 8 8 1 4 8 18</speed>
      </Wind>
      <Geomag model="IGRF-11">
        <decl>-1.7506 -1.8492 -1.5492 -1.1824</decl>
        <ddecl>0.0354 0.0355 0.037 0.04</ddecl>
      </Geomag>
      <temp>15.9 14.8 14.6 13.3 -29.5 -29.7 -29.8 -30.4</temp>
    </Attributes>
  </Response>
</nwx>

Note how this shorter version has 81% and 78% smaller request and response sizes, respectively, albeit the extra data may cause the processing time to be slightly higher.
This also illustrates how the data are ordered when both extended p and z are used: all the data for the first altitude level comes first, then the next altitude level and so on.

Conclusion

Taking advantage of the new semantic possibilities offered by version 0.3.5 can help you reduce your bandwidth requirements, something which I expect will be welcome by users of mobile applications and everyone on a slow connection.

Version 0.3.5 is now released, with the following changes:

• Implemented the NwxData generic response format, as previously announced. This is now supported by the <ProfileGraph/> request.
• Heavier caching for <ProfileGraph/> requests. Avoids generating duplicate profiles when the same data is requested more than once. In those cases, only one of the requests will do the actual processing while the other(s) will sit there waiting for the first one to finish, then all return the same graphic.
• The server will now refuse to process new requests if it is overloaded. CPU intensive requests such as <ProfileGraph/> and <Chart/> will be rejected first, and if the situation continues, every new request will be rejected via a code 15 <error/>; as an eventual last step any ongoing requests will be ungracefully aborted (TCP connection closed).
• Implemented the u attribute for <AvailableLevels/> requests.
• Implemented the z extended attribute.
  You can now retrieve data for multiple altitudes in the same request in the same way as you used to be able to query multiple horizontal positions via the p extended attribute. Both of them can be combined in the same request, in which case the data is returned in position/altitude order.
• Now all requests, except <Metar/> and <Taf/> accept extended geographic positions and where applicable, the extended z attribute.

Work is being done on improving the <ProfileGraph/> request with the aim of giving developers more choice in the ways they can use NWX data.

The latest changes focus on providing applications with raw numeric data that they can use to create or augment their own profile graphics, amongst other things. This will be done via the addition of a new supported MIME type: application/x-nwxdata+xml, which will return data using a simple and relatively compact custom XML schema.

In order for interested developers to provide feedback on this new feature, a pre-release version of the development code is being made available as 0.3.5a1 (<nwx version="0.3.5a1">), and the NWX Documentation has been updated with details on these additions. For more information, please see the What's New section for 0.3.5.

Version 0.3.4 is now released, with the following changes:

• New <Attributes/> container element, used to define one or more attributes which will be applied to its children; it enables client applications to express certain requests in a more compact form. Thanks to Philipp Matzinger from the goVFR team for the suggestion.
• GIF and JPEG image output formats supported by the <ProfileGraph/> request. Thanks to François Fouchet for proposing this addition.

The list of available geomagnetic models has been expanded to include the latest iterations of the International Geomagnetic Reference Field (IGRF-11) and World Magnetic Model (WMM2010). Both are valid through 2015, and the change applies to all NWX versions supporting the <Geomag/> request.

As of three days ago, the NWX forecast reaches all the way out to T+108 (i.e., up to four days and six hours in the future), and up to the 200 hPa level (approx. FL380) while adding additional pressure levels at 800 and 600 hPa.

As had been mentioned previously, at one point the bulk of the METAR and TAF archives had to be taken offline as I had to reclaim the storage space for extending the weather forecast. Although still undergoing validation, I expect that I have now finally come up with a long-term solution by moving the archival database to another server better suited for the task and as a result of this change, the archive is now back online.

The entire dataset reaches back to noon UTC on 2008-11-06 and currently holds over 75 million METAR observations and over six million TAFs for reporting stations worldwide. Historical reports can be accessed on the WWW at my METAR data access page. Currently there is no convenient way to search for historical data via NWX, but that functionality can be easily implemented if requested.

I have done a very minor but hopefully significant update to the NWX reference documentation. The change consisted on making sure all the examples show a current epoch (e attribute) and the NWX version attributes reflects a recent iteration of the code.

The reason for this change is that I regularly keep seeing entries in the NWX error log which appear to indicate people are copying and pasting the examples directly into an HTTP request, which throws back a (sometimes cryptic) error about the requested epoch not being valid (as it's well in the past and thus not part of the current forecast cycle).

It is not at all unreasonable to expect an example given in a system's documentation to just work however, hence why I have made this simple change. As a result, from now on and unless it's clearly and explicitly mentioned, examples copied and pasted from the documentation should return a valid result, although the actual result values probably will be different than what's shown in the docs.

I'm sure there are many other areas where the documentation could be improved, and hopefully this is a useful start. Needless to say, any suggestions as to aspects which might be poorly explained are more than welcome.

Version 0.3.3 is now released, with the following changes:

• New <srss/> request, giving the times of sunrise and sunset.
• Added <theme>route</theme> to <Chart/> request.
• Improved <ProfileGraph/> request, incorporated the previously discussed SVG support and data caching for better response speeds in certain cases.

Those of you who need or wish to ensure the integrity and in-transit confidentiality of NWX requests may do so by accessing the service over HTTPS. The only change required to achieve this is to change the canonical URI from http://navlost.eu/aero/nwx to https://navlost.eu/aero/nwx.

Client authentication via X.509 certificates (instead of or in addition to the mechanisms provided by the NWX protocol itself) can also be implemented upon request.

As part of the push to improve the generation of vertical flight profiles, a number of modifications have been done to the profile engine, the goal of which is to reduce profile creation time for repeated requests.

Essentially this works by caching any data previously requested within the same forecast cycle, rather than looking it up again in the weather model files which is a much more expensive operation. Thus, if for example you have planned a route and notice it'll take you right through all the icing, and decide to try again by changing flight levels, or perhaps doing a small detour, the profile engine will reuse as much data as it can from any previous requests, retrieving only whichever data is not already in the cache.

In the most extreme case, which is replanning the same route at the same time but at a different flight level, in my tests profile creation time has been reduced from 30 seconds down to 1.5 seconds (a 20× speed-up).

While I have some concerns about the scalability of this approach, this can currently be tested by using a service version number of 0.3.3-pre2 and is planned to be included in the final 0.3.3 release. This is of course available for both PNG and SVG outputs (see my previous announcement)

As usual, feel free to get in touch with any questions or comments.

Work is ongoing on SVG support in the <ProfileGraph/> request.

The idea behind this is that SVG is able to offer better control over style and presentation and allows for better integration with the client application's own environment. For instance, you have complete flexibility to change the chart's presentation by applying your own stylesheet, doing an XSL Transformation, or even via DOM-level manipulation.

The goal is to have some kind of SVG support by the next revision of the NWX code, for which I haven't set a date yet, however, I will publish a number of pre-release versions as needed so that you can test and assess the level of SVG support your platform offers, make suggestions, etc. For the moment, version 0.3.3-pre1 is available for testing, and already has a working SVG output.

The way to get an SVG graphic is by means of the new mimetype="image/svg+xml" attribute to <ProfileGraph/> (conversely, mimetype="image/png" produces a PNG file, which is the default). The SVG can be returned in-line as usual, in which case it is part of the NWX reponse document, or out of band, in which case it is a stand-alone XML document. Presently, it comes with a default, hard-coded stylesheet and the <colour/> styling attributes are not honoured, but that's going to change as work progresses.

Here is an example of a normal PNG output, and the corresponding SVG version (also saved back as PNG for those without proper SVG browser support).

I hope this will turn out to be a useful feature, and as always, comments are appreciated.

Version 0.3.2 is now released, with the following changes:

• <temp/> requests are now fast, and therefore, properly usable. It now takes an extended p attribute, and the altitude (z and u) attributes are optional: a request without a vertical coordinate will return the forecast ground level temperature at that point.
• A new request, <qff/> has been added. It returns forecast mean sea level pressure—please note that this is not QNH.

METAR and TAF reports

Take advantage of the position attribute.

When making a TAF or METAR request, sometimes one may be interested not so much on weather reports for a specific airfield as for a general area, and it might be that the airfield we want data for does not have meteorological facilities or is not reporting (e.g., it might be open only on certain days).

Often, however, we can use data from another reporting station nearby. If your application has access to a navigation database (as I expect most flight planning apps would), then instead of making an ICAO-based request, your could simply send the WGS84 coordinates for your target. Consider this example for Paray-le-Monial (LFGN), a small airfield in the Burgundy region of France:

Example request, using icao attribute:

<nwx version="0.3.1">
  <Request id="1">
    <Metar icao="LFGN" maxage="3"/>
  </Request>
</nwx>

Response:

<nwx version="0.3.1">
  <Response id="1">
    <Metar icao="LFGN">
      <warning code="11">LFGN: No data found</warning>
    </Metar>
  </Response>
</nwx>

which is expected since Paray does not have weather reporting.

However we could try this instead, using the p attribute:

<nwx version="0.3.1">
  <Request id="1">
    <Metar p="46.4667,4.1331" maxage="3"/>
  </Request>
</nwx>

Response:

<nwx version="0.3.1">
  <Response id="1">
    <Metar p="46.417,4.017" dist="5.7" icao="LFLN">
      <report epoch="2009-11-04 14:00:00">
        <![CDATA[LFLN 041400Z AUTO 25009KT 9999NDV SCT044 BKN056 OVC078 12/05 Q0998]]>
      </report>
    </Metar>
  </Response>
</nwx>

which returns the latest report for Saint Yan, an IFR airfield only six nautical miles away (as indicated by the dist attribute in the response).

Requesting Charts and Profile Views

Use a separate connection for graphical data.

While typical data requests such as wind speed and METARs are pretty fast, that is not the case in general for graphical information: charts take 4.5 seconds on average to generate, and profile views take an average of 30 seconds.

One thing I have noticed from my own applications is that, if both numerical and graphical data are included on the same <nwx></nwx> request it can make the application feel sluggish while it sits there waiting for the server to start pumping data back.

Fortunately there is an easy solution: you can open up two separate connections to the server. Group all the "fast" data on one (METARs, wind, magnetic declination, etc.), and put all the "slow" stuff in a second connection. The server will process those in parallel and return the first batch of data as soon as it's available, so it can be displayed in your application, giving the user something useful to look at while he waits for the graphs.

It could also help if you are requesting both charts and profile views, to put those on separate requests as well. Also consider showing some sort of placeholder or busy indicator while the profile view loads—obviously it won't actually load any faster, but it will make the application appear more responsive.

Version 0.3.1 of the NWX service is live as of today. This contains only minor bug fixes to 0.3.0, namely:

• In the <Chart/> element, <theme>setmpdraw on</theme> would not display in most cases, this has now been fixed.
• If given invalid image dimensions for the <Chart/> or the <ProfileGraph/> elements, an assertion would be triggered, causing the code to abort processing. As of v0.3.1 an <error/> element is returned instead.

The next release, which will introduce a few more bug fixes and a couple of new features, is scheduled to be released in late November. Please use email or the contact form if you have any feature requests, bug reports, or general suggestions.

The network infrastructure upgrade originally scheduled for 21-23 Oct has been moved to the end of this week. Work will be carried out between 0000Z and 0600Z starting on 28-Oct and ending on 30-Oct. An estimated 90 minutes of cumulative service outage is expected during this period.

We apologise for the inconvenience. Please visit http://navlost.eu/contact if you require more information.

The NWX reference document at http://www.navlost.eu/nwxs/doc/help has been updated to correct a few typos. Two of those were in the examples, namely:

• The section describing the <application/> element, presented one of the element's attributes as version when it should have read instance.
• In the section describing the <ProfileGraph/> element, the <Weather/> subelement was spelled with a lowercase 'w', whereas an uppercase 'W' is required.

If you discover any more typos, thanks for reporting them to me—if an example doesn't work after you've set a valid epoch, chances are there is a typo in the docs so don't hesitate to drop me an email.

The facility hosting the navlost.eu server will be undergoing an upgrade of their network infrastructure. Work will be carried out between 2300Z and 0500Z starting on 21-Oct-2009 and ending on 23-Oct-2009. An expected 90 minutes of cumulative service outage is expected during this period.

We apologise for the inconvenience. Please visit http://navlost.eu/contact if you require more information.

What's New

After quite a bit of heavy coding over the past few days, finally the latest version of the NWX Service is ready for public access.

There have been some significant changes to the way the data is handled and processed, with the primary aim of making things more efficient in terms of storage space and CPU, while maintaining the reasonably short response times. A secondary aim still in the works is to reduce those response times for some of the slower requests (e.g., temperature data).

As a result of all this, these new features have been added:

• A new pressure level. Now you can get winds and chart data all the way up to 300 hPa (around 30000 feet).
• New forecast epochs. Now the forecast goes all the way out to T+96, which is just short of four days in the future. As far as winds are concerned, it's questionable whether an estimate four days in advance will be any good, but nevertheless this extended forecast might still be useful for getting an idea of developing weather patterns.
  Note that this enhanced forecast range is only available if you are using v0.3.0 or newer.
• New turbulence plots. This highly experimental feature calculates the likelihood of clear air turbulence and makes the result available as an additional layer in the <Chart/> request.
  This only predicts clear air turbulence (primarily related to the jet stream)–neither orographic turbulence, or storm-related turbulence are depicted. See this previous entry for more information.
• Another improvement on this version is the earlier availability of new forecast cycles. The forecast is updated four times a day: the process starts nominally at 00:20, 06:20, 12:20, and 18:20, and attempts to retrieve the latest cycle which is typically about six hours old by that point (e.g., the 12:20 update cycle loads the 06:00 forecast, which is generated sometime between 10:00 and 12:00).
  With the old wind database, it normally takes about one hour and forty minutes for all of the wind data to be refreshed, but with the new method the updated wind information is available in less than thirty minutes–and that's 31 epochs, rather than the previous 17.
• More as a side-effect of the ongoing upgrades than anything else, the wind data resolution has been improved and now the new method uses the full half-degree resolution of the native model, while the previous data used a downsampled two-degrees grid. In most cases this is hardly of any practical consequence, and it comes free with the package anyway, but having said that, it should improve things a bit for people flying in mountain areas.
  Note also that an implication of this change is that you will get slightly different wind values if you use a pre-0.3.0 interface compared to the newer versions. This is normal and expected.

Except for the additional 300 hPa pressure level, none of the other changes have been backported to previous versions of the service, so this might be a good opportunity for application developers to consider upgrading to a post-0.3.0 version.

At the moment, both the new and old wind databases are running in parallel, so as to have a fallback if any serious issues creep up with the new implementation, but eventually the old one will be stopped for good. If by then everyone has moved to a newer version, and depending on the number of legacy applications still running in the wild, and the responsible developer's ability to have them upgraded, I'll make a decision as to whether to retire some of the older (i.e., <0.3.0) versions of the service. In any event, I'll give everyone at least a year's notice before any old versions are stopped–just make sure you check this space once every few months!

What's Broken

In order to keep the old wind service running alongside the new one, something had to give in, and that turned out to be the METAR archive.

METARs and TAFs from 2009-10-01 onwards are still available via the usual interfaces (Web, and NWX). Older METARs (back to late March 2009) are still there but not currently accessible. Although I can make them accessible in a very short time upon request, the query response times will be in the order of two to three minutes for each request referencing a date prior to October 2009.

In any event, this is only a temporary solution until I find a better approach to handle the storage requirements for this archive more economically. The two solutions I am mainly considering at the moment are a) off-site storage (relative to the server), and b) a compression scheme which would allow me to compress each METAR individually while still taking advantage of redundancy across reports.

Recent METAR and TAFs (i.e., less than a few days old) are and will stay available as normal. I have sacrificed the archiving feature as it did not seem to get much use (except for the odd queries from NOAA and UKMO addresses... I would have thought they would have their own archive?), however, if anyone has a good use case, let me know and I'll make this higher priority.

What's Next

Apart from consolidating the recent improvements, I'll try to get the Blitzortung sferics gateway in place, as mentioned in an earlier article. Other than that, I am open to suggestions as always. Feel free to get in touch if there are any particular features you would like to see.

During the course of today (16-Oct-2009) some server maintenance and testing will take place, with the principal aim of making the service more resilient to upstream failures, by giving the server the capability to react to some of those failures when possible. E.g., if an upstream server is down or stale, it will automatically switch to a backup provider.
This is in addition to the automatic external monitoring already in place, which alerts me via email and instant messaging if there are any problems, such as the NWX server going down.

In addition to the above, there are plans for an increase in server capacity which I'll be testing today. The plan is to add another pressure level, thus giving weather information all the way up to about FL300, and also extending the forecast range–initially this will be by another twelve to eighteen hours, but the ultimate aim is to all the way out to four days.
What this means however, is that if anyone out there is making any assumptions as to the number of epochs or pressure levels available, this will be a good time to rethink your approach, as those can and will change.

A new and very experimental feature has been added to the NWX service, consisting of a clear air turbulence prediction index. This is an implementation of the original DVSI index described in [Ellrod and Knapp, 1992], based on the 0.5 degrees GFS at the 400, 500, 700, 850, and 1000 hPa levels. Currently this is generated at 12-hour intervals starting from forecast epoch T+06, and the display consists of a rainbow colour scale for the raw index (typically taking values between zero and ~15-20). The exact meaning of the index in terms of turbulence severity is subjective and dependent on the index's realisation method, so this will need to undergo some testing and comparison with aviation forecasts from official sources before this can be translated into a "light, moderate, severe" type of scale.

Partly as a result of this addition, a number of significant improvements have taken place in some areas of the service, although the only user visible changes (apart from the addition of the turbulence index) are slightly new semantics for the <AvailableEpochs/> and <AvailableLevels/> requests (addition of an attr attribute, a downward-compatible change), and the removal of the <href> element in the <Chart> reponse (a downward-incompatible change, but I don't think anyone was using that element anyway), which is replaced and improved by the output attribute.

All this is expected to appear in the next version 0.3.0, which will be available within the next day or two.

Version 0.2.5 of the NWX Service is now live. It incorporates the following improvements:

• Magnetic declination can now be requested from the service, via the <Geomag/> call. It supports two magnetic models: IGRF-10 and WMM2005. Newer models will be added in due time.
• A new redirect option has been added to the out-of-band output options which were implemented (although not publicised very much) back in version 0.2.3. Out of band output is currently possible for <Chart/> and <ProfileGraph/> requests, and the plan is to add it to any new requests with the potential to return large binary payloads.

Application Keys Rollout

As the NWX service grows, it is starting to become necessary to have a means of keeping track of usage so that everyone can get a satisfactory experience.

To this end, a registration system will be put in place, very roughly along the lines of Google Maps where you get a code that gives your application access to the service. This is in addition to individual user subscriptions, which address a different aspect of the service: an API key will give your application access to the service, while user authentication via the <user> element could give your user access to additional data or personalised information.

Final implementation details will be made available on the NWX help page with sufficient notice so that developers can adapt their applications as required. After rollout, keyless access will still be possible but it will be restricted to a small number of requests per IP address per day, so as not to inconvenience low-volume users and to allow developers who wish to try out the system to do so without needing to request an application key first.

Sferics Data

An accord in principle has been reached with sferics community Blitzortung.org to redistribute their sferics data via NWX. This agreement would allow users who participate in the Blitzortung network to access lightning strike data from their NWX-enabled applications, thus complementing the forecast-based information with accurate near real time data.

If you wish to receive sferics data but are not yet a Blitzortung member, you will need to set up a lightning detector (essentially a specially made VLF receiver) and forward their data to the Blitzortung server. The only cost involved for the user is that of setting up the receiver (between €100 and €200). There are no access or maintenance fees of any kind being levied either by Blitzortung or by Navlost.eu.

For more information on how to participate in the lightning detection network, please read the following article.