# Changelog All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ## [Unreleased] ### Added - Added spack package manager support - Added Code of Conduct - Added message passing layer base type - Added convolutional MPNN layer - Added framework for residual blocks - Allow layers to take in multiple inputs - Added handling for multiple inputs - Added skip layers - Added internal version and build information - Collapsed inputNd_layers to a single input_layer type - Collapsed flattenNd_layers to a single flatten_layer type - Added graphstruc external library dependency - Added hyperparameter setting procedure for each layer - Added custom array derived type - Added conv1d_layer_type - Added avgpool1d_layer_type - Added maxpool1d_layer_type ### Changed - Updated Contributing document - Changed module filenames to improve consistency and reduce conflicts - All modules have "athena__" prefix - Handle errors consistently - Changed mod_types.f90 to mod_misc_types.f90 - Changed athena__infile_tools to athena__tools_infile - Renamed real12 to real32 - Make generic procedure for pool_layer printing - Remove forward and backward passes from layer container - Changed read procedure for layers - Split into two procedures - One that is a procedure contained in the layer derived type, with the layer as the parent - One that is a public procedure that returns the layer from the read - Changed layers outputs and gradients to derived type to enable generic handling - Changed commenting in all Fortran files to adopt FORD compatible documenting - This enables documentation of the modules to be compilable using the FORD code (https://forddocs.readthedocs.io/en/stable/) - This tries to follow the format outlined by the Fortran Lang style guide (https://fortran-lang.org/learn/best_practices/style_guide/) ### Fixed - Fixed unallocated lr_decay reference - Fixed indentation in mod_misc_ml.f90 - Fixed identity initialiser - Fixed indentation in mod_misc_ml.f90 - Fixed reference to fpm install in README - Fixed unallocated lr_decay reference - Fixed ifort and ifx compiler reference to temporary arrays - Fixed temporary array reference in tests - Fixed epsilon undefined value in mod_loss - Fixed project link in CMakeLists.txt # [1.3.3] - 2024-07-03 ### Changed - Updated license year ## [1.3.2] - 2024-06-28 ### Added - Added conv1d_layer_type - Added avgpool1d_layer_type - Added maxpool1d_layer_type - Added input2d_layer_type - Added statement of need to README ## [1.3.1] - 2024-05-25 ### Changed - Add reference to gcc 12.3 compatibility in README ### Fixed - Fix attempt to assign size of random seed in random_setup - Fix attempt to assign size of random seed in test_network - Fix attempt to assing size of random seed in example/simple - Fix abstract interface use in base_layer_type - Add appropriate references to neural-fortran in CONTRIBUTING.md and associated procedures, and modules - Fixed gradient clipping not being used in update or get_gradients procedures of network_type ## [1.3.0] - 2024-03-13 ### Added - batchnorm1d_layer_type (handles both 1D spatial channel data and rank 0 channel data) - base_layer submodule file - network submodule file - Added comments to network and base_layer procedure interfaces ### Changed - Improved comment and headers in all source files - Improved comments in all test source files - Added interfaces within base_layer submodules for ease of readability - Added interfaces within network submodules for ease of readability - Moved methods of base_layer into submodule - Moved methods of network into submodule ### Fixed - Remove reference to doc/ directory - Fixed typo in example library - Fixed example/mnist/src/main.f90 using wrong constants module - Fixed cmake isntall path prefix duplication issue ### Removed - Deprecate and remove step_decay and lr_reduce_on_plateau procedures ## [1.2.4] - 2024-03-07 ### Added - fpm continuous integration workflow ### Fixed - move location of output rank test in maxpool3d test ### Removed ## [1.2.3] - 2024-03-07 ### Added - More unit tests - Code coverage reporting ### Changed - Unify shuffle and split procedure names - Change handling of magnitude optional argument in renormalise_sum - Convert copy and reduce to child procedures of network_type - Set already_initialised to False by default for random_setup ### Fixed - Index reference in initialiser_ident - avgpool3d forward pass from max to sum - avgpool3d output shape initialisation - Correct handling of "valid" padding method - Random seed for shuffle - conv2d input gradient calculation - conv3d input gradient calculation - Incorrect set_gradient spread dimension - MAE total loss using MSE instead of MAE - metric_dict allocation handling error - icount field separator - Optional argument handling in renormalise_norm ### Removed - Remove assign_list procedure from mod_tools_infile - Remove assign_listvec procedure from mod_tools_infile - Remove cat procedure from mod_tools_infile ## [1.2.2] - 2024-02-18 ### Added - Add fpm compilation handling - Add status badges to README ### Changed - Move example-dependent library from /example/include to example/example_library - Update README to include fpm documentation - Update README to include dependencies - Migrate repository from GitLab: https://git.exeter.ac.uk/hepplestone/athena - Migrate repository to GitHub: https://github.com/nedtaylor/athena ### Fixed - Fix typos in README - Fix typos in CHANGELOG ### Removed - Remove parallel reference from examples ## [1.2.1] - 2024-02-07 ### Changed - Change CMakeLists to break on parallel build - Tidy up TODO ### Fixed - Fix typos in README ### Removed - Remove parallel build option - Remove src/athena_omp.f90 - Remove superfluous comments and urls ## [1.2.0] - 2023-12-18 ### Added - Add inference to dropout layers - Add 2D and 3D batch normalisation layers - Add 1D and 4D flatten layers - Add predict procedure to network_type - Add network reset procedure - Add Adam optimiser - Add procedures to get and set network parameters - Add procedures to get and set network gradients - Add procedure to get layer output - Add new test directory - Test jobs test the working of the library - Add test jobs - Add more example jobs - Example jobs show the functionality of the library - batchnorm, dropblock, 3D, sine, and simple function - Add average pooling 2D and 3D layers - Add RMSprop, Adam, and Adagrad optimisers - Add learning rate decay - Allow accuracy scoring methods to be imported ### Changed - Handle batch size in each layer - As such, increase rank of io data (and gradients) for all layers by 1 - Handle loss and metric network initialisation in separate procedures - Convert optimiser to parent and derived types - Change optimise procedure to minimise - Handle network learning in network, rather than individual layers - Rename test/ to example/ - network_tyep%forward now handles assumed rank - input_layer_type%set now handles assumed rank - Add abstract conv, pool, and batch layer types - All layer forward and backward passes take defined shape - Previously, rank was defined and shape was assumed - Flatten layers now use output_shape as well as num_outputs - Make accuracy score functions available to import ### Fixed - Fix drop inference - Fix backpropagation for overlapping windows in maxpool layers - Fix dropblock 2D and 3D generate_mask to check for .lt. gamma instead of .gt. - This was already correctly handled in dropout - Fix R2 score function - Add zero division check ### Removed - Remove mod_batch_norm.f90 - Remove update procedure from learnable_layer_types - Remove parameter velocities from learnable_layer_types ## [1.1.3] - 2023-11-26 ### Changed - Update argument consistency for split and shuffle procedure ## [1.1.2] - 2023-11-24 ### Fixed - Parallel extent of y_true in test procedure - Define language and compiler in CMakeLists.txt for parallel build type ## [1.1.1] - 2023-11-24 ### Changed - Improve README ### Fixed - ifort and ifx compatibility ## [1.1.0] - 2023-09-14 ### Added - Dropblock layers - Padding procedure - Split procedure - Loss methods for continuous data - Test example ### Changed - Convert layer types to polymorphic - Allow polymorphic output data ## [1.0.0] - 2023-09-12 ### Added - Convolutional layers - Full/dense layers - Shuffle procedure - Network model derived type ### Changed - Convert to compilable library ### Removed - Program format of athena