As of July 5th, operational data from the AWS instances has been made available for testing at ftppro.knmi.nl. This upcoming update indicates that this data will now be transmitted to the operational FTP server, ftpservice.knmi.nl, and data from our previous processing system will no longer be shared. In the coming weeks, the old processing system will be fully decommissioned.
The upcoming change signifies that EMADDC will cease operating a complete DTAP (Development, Testing, Acceptance, Production) set of systems. Instead, we will utilize separate environments for development and acceptance (DTA) and production (PRD). EMADDC will consistently maintain three environments: acceptance (acc), production (prd), and production-2 (prd-2). The acceptance and production environments will be utilized for the standard EMADDC products, while the production-2 environment will be exclusively designated for EMADDC MetOffice Global products.
Previously, the old system facilitated communication between DTAP environments, which led to challenges such as limited data availability in 2023. This issue is now resolved through the implementation of distinct environments. Additionally, each Lift and Shift system/environment will operate with its own dedicated database, resulting in reduced costs, enhanced flexibility during deployments, and improved performance. New monitoring dashboards have been introduced to promptly alert EMADDC of any potential issues. Moreover, the process for new software releases has been streamlined, allowing for deployments with minimal impact on users, typically involving only a brief downtime of several minutes.
This indicates that EMADDC is expected to implement updates with greater frequency in the future. Consequently, we have revised and streamlined our versioning system. The software version will now adhere to the following template:
Software Version: Major Release.Minor Release_Machine Letter
The current operational version of EMADDC running on the production envrionment is 2.2.115_p. Release 2.2 is anticipated to receive only minor updates until the rollout of EMADDC 3.0. It is essential to understand that modifications in Major Releases (reflected in the 2.2 portion of the version) signify substantial changes to the system that may affect the quality of observations or result in other significant implications. In contrast, the Minor Release (indicated by the .xx portion of the version) will undergo regular updates, incorporating new code, fixes, or adjustments to features. The machine letter denotes the environment utilized for product creation, with files available at ftpservice.knmi.nl from our production environment and files accessible at ftppro.knmi.nl from acceptance environment once this update has been deployed.
Numerous minor enhancements and adjustments have been implemented to improve the robustness of the processing system. One notable change that users may observe is that the observation ID is now reset on an hourly basis rather than daily.
The file formats have remained consistent, and the quality of observations continues to be unchanged. Aside from the revised versioning scheme and observation ID changes, users of EMADDC products should not anticipate any additional impacts.