Development and release processes#

Definitions#

  • PyGranta meta-package: A package which depends on the idiomatic packages and auto-generated packages. Includes documentation describing the PyGranta ecosystem, including this document.

  • Idiomatic package: A hand-written package. Typically depends on one auto-generated package and openapi-common. There are currently three idiomatic packages in PyGranta:

    • ansys-grantami-bomanalytics

    • ansys-grantami-recordlists

    • ansys-grantami-jobqueue

  • Auto-generated package: A package generated by the codegen template. There are currently two auto-generated packages in PyGranta:

    • ansys-grantami-bomanalytics-openapi

    • ansys-grantami-serverapi-openapi

  • Codegen template: The template used to generate the auto-generated packages.

  • Production test VM: A virtual machine used as part of CI. A new Production test VM is commissioned for each new Granta MI release, and the VM is only upgraded if a service pack is made available for that release. It is used as part of idiomatic package CI during development and release to ensure compatibility with a specific version of Granta MI.

  • Development test VM: Virtual machine used as part of CI. This machine is upgraded as required during development. It is used as part of idiomatic package CI during development only to ensure compatibility with development builds of Granta MI.

Development and release timeline#

The timeline is split into three distinct phases:

  1. Phase 1: Development. There is active development upstream on closed-source Granta MI components, the codegen template, and on the open source idiomatic packages and auto-generated packages. Effort is made to ensure CI is passing on all main branches, but from time to time CI may fail. The development test VM is upgraded as required to support CI passing on main branches.

  2. Phase 2: Hardening. Bugs are fixed and minor improvements are made to idiomatic packages, auto-generated packages, and the codegen template. During this phase the codegen template and auto-generated packages are finalized.

  3. Phase 3: Pre-release. The upstream closed-source development has completed. The PyGranta meta-package and idiomatic packages are manually tested and finalized.

Phase 1: Development#

During development, the following processes happen continuously:

  1. Incorporate changes to upstream API definitions into auto-generated package repositories.

  2. Check that code generation is successful by monitoring CI in the auto-generated package repositories.

  3. If code generation is not successful, resolve by either fixing the codegen template, fixing the auto-generated package configuration, or raise the issue with the internal Ansys development team.

If idiomatic package development does not depend on upstream API development, then there is no dependency between the idiomatic package and the auto-generated package and work can proceed in parallel.

If significant idiomatic package development is being done that depends on the current state of an auto-generated package, you should update the main branch of the corresponding idiomatic package to depend on the auto-generated package main directly.

During this phase, the development test VM may be upgraded if required to support new upstream Granta MI capabilities.

Phase 2: Hardening#

Phase 2a: Initial dev releases#

  1. Update the development test VM with the latest Granta MI software and databases. See the ansys-grantami-bomanalytics repository for details about preparing the Restricted Substances test database.

  2. Publish auto-generated packages to the internal PyPI mirror with .dev version numbers.

  3. Update idiomatic package main branches to depend on the auto-generated package .dev releases. If the main branch has not been updated to depend on the auto-generated package main branch during development, then there may be integration issues discovered at this point - these should be resolved in the PR before merging. The PR typically includes:

    • Updates to Python version support to align with changes to the auto-generated package.

    • Updates to the README.rst to reflect any changes in the minimum required Granta MI version.

    • Changes at the interface with the auto-generated package if either the codegen template or API definition has changed.

    • Changes to expected test results. This is typically relevant for ansys-grantami-bomanalytics which includes acceptance-style tests.

  4. Publish idiomatic packages to the internal PyPI mirror with .dev version numbers.

    Note

    At this point release branches should not be created. .dev releases should be published to the internal PyPI mirror from the main branch.

    Note

    These .dev releases can be used for internal testing of closed-source packages that have runtime dependencies on PyGranta packages.

  5. Continue to review any changes to API definitions via auto-generated packages throughout the hardening phase. If there are any issues with the API definitions, these must be escalated with high priority.

Phase 2b: Documentation review#

Review the documentation for all idiomatic packages and the PyGranta meta-package:

  • Significant new documentation is reviewed by the appropriate reviews for linguistic and technical accuracy.

  • References to dependencies are updated, including:

    • Server software versions

    • Required Python versions

    • Compatible third-party library versions

  • References to Ansys documentation of other packages/products are checked to see if they need to be updated to the previous version of that documentation.

    Note

    The Ansys documentation for the current in-development release is published after the PyGranta packages are released. The documentation for the previous release should be used to ensure links in published PyGranta documentation can always be resolved.

Phase 2c: Release candidates#

Towards the end of the hardening phase, once the API definitions are determined to be stable, the auto-generated packages are released:

  1. Create release branches for auto-generated packages and publish as release candidates to public PyPI.

  2. Create Pull Requests in idiomatic package repositories to update main to depend on the auto-generated package release candidate releases. Ensure CI passes for idiomatic packages.

  3. Publish auto-generated packages as stable releases to public PyPI.

  4. Create Pull Requests in idiomatic package repositories to update main to depend on the auto-generated package stable releases. Ensure CI passes for idiomatic packages.

  5. Create release branches for idiomatic packages and publish as release candidates to public PyPI.

  6. Update the PyGranta meta-package main branch to depend on the public idiomatic packages release candidates and auto-generated packages stable releases.

    Note

    At this point release branches should be created for idiomatic packages. However, during hardening these release branches still run CI against the development test VM.

    Note

    Idiomatic package release candidates must depend on stable release versions of auto-generated packages only. They must not depend on pre-release versions of auto-generated packages.

    Note

    When performing pre-releases of packages with towncrier, the Ansys action creates a Pull Request to update CHANGELOG.md on main, and commits the CHANGELOG.md changes directly to the release branch. The commit to the release branch should be reverted, and the Pull Request to merge the changes to main should be closed and not merged.

The stable auto-generated packages and release candidate idiomatic packages can now be used for the final release of internal packages. It is recommended to make any closed-source idiomatic package release candidate dependencies flexible enough such that they match subsequent stable releases of those idiomatic packages.

At the end of this phase, the PyGranta meta-package depends on release candidates for the idiomatic packages and releases of the auto-generated packages. This guarantees that there exists a consistent set of dependencies across all idiomatic packages and auto-generated packages. Any issues at this stage should be resolved by addressing any dependency conflicts in the idiomatic packages and publishing new release candidates to public PyPI as necessary.

Checklist#

  • The development test VM is running an up-to-date development version of Granta MI.

  • CI for all idiomatic packages is passing.

  • Documentation for all idiomatic packages and the PyGranta meta-package has been reviewed to verify significant new content, changes in dependencies (for example Python version support, minimum Granta MI version), and references to other Ansys documentation sources.

  • release branches have been created for auto-generated packages and idiomatic packages.

  • Stable auto-generated packages are available on PyPI.

  • Internal packages with runtime dependencies on auto-generated packages depend on the stable PyPI releases.

  • Idiomatic packages depend on stable releases of auto-generated packages only.

  • Release candidate idiomatic packages are available on PyPI.

  • Internal packages with runtime dependencies on idiomatic packages depend on the release candidate PyPI releases.

  • The PyGranta meta-package main branch depends on idiomatic packages release candidates and auto-generated packages stable releases only.

Phase 3: Pre-release#

  1. Provision a new production test VM to run the Granta MI release validated during the hardening phase.

  2. Update the latest idiomatic package release branches created during the hardening phase to run CI against the newly provisioned production test VM.

  3. Update the latest idiomatic package release branches created during the hardening phase to not run CI against the development test VM.

  4. Manually test the PyGranta meta-package against the Granta MI release candidate by installing the main branch directly using pip.

Note

If there have been significant changes to the PyGranta meta-package, a release candidate may optionally be published to PyPI. Examples of significant changes include re-organization of the documentation and changes to the CI process, both of which are not checked as part of the manual test required in this phase.

Testing should occur within 4 weeks following Phase 2. Any issues raised during testing should be immediately triaged and fixed or deferred.

Checklist for completing this phase:

  • There is a new production test VM provisioned and running the release version of Granta MI.

  • All release branches no longer target the development test VM.

  • CI for all idiomatic packages is passing.

  • Manual testing has been completed against the PyGranta meta-package and all idiomatic packages.

  • The GitHub PyGranta Development project contains no must-deliver features or bug fixes still outstanding for the active release.

Phase 4: Release#

The PyAnsys meta-package is typically released a week before Granta MI is available to customers. As a result, the stable versions of the packages should be released at least a week before Granta MI is available to customers:

  1. Defer any issues still associated with the active release in the GitHub PyGranta Development to a future release.

  2. Publish idiomatic packages as stable releases to public PyPI.

  3. Update the PyGranta meta-package main branch to depend on the public idiomatic packages stable releases.

  4. Create (or update if a release candidate was published in Phase 3) a release branch for the PyGranta meta-package.

  5. Publish the PyGranta meta-package as a stable release to public PyPI.

  6. Create a Pull Request in the PyAnsys meta-package repository to move the PyGranta meta-package dependency to the new stable release.

  7. If any new idiomatic package releases are compatible with the previous release of Granta MI and PyGranta, including the minimum supported Python version, create a patch release of the previous PyGranta meta-package release which includes the new idiomatic package releases.

Checklist for completing this phase:

  • The active release in the GitHub PyGranta Development project is empty, with all deferred tickets re-assigned to a future release.

  • Stable releases of all idiomatic packages and the PyGranta meta-package are available on PyPI.

  • The PyAnsys meta-package depends on the PyGranta meta-package stable release.

  • A patch release has been made for the previous PyGranta meta-package if there is at least one backwards- compatible idiomatic package released during this development cycle.

Addenda#

Test VM management#

There are multiple production test VMs in use at one time, and each runs a validated Granta MI version. The names and URLs of the production test VM are stored in the AZURE_VM_NAME_yyRn and TEST_SERVER_yyRn_URL secrets respectively, where yyRn is the Granta MI version, for example 25R2.

Production test VMs are commissioned for a new Granta MI release once it is declared stable, and is decommissioned when support is withdrawn for that Granta MI release.

The development test VM runs a development build of Granta MI, and is upgraded as required to support CI on idiomatic package main branches during development. The name and URL of the development test VM are stored in the AZURE_VM_NAME_DEV and TEST_SERVER_DEV_URL secrets respectively.

The development test VM has a green desktop background to indicate it is safe to upgrade.

This approach has the following advantages:

  • CI is generally passing on all idiomatic package main branches at any point in development and release, which reflects successful CI run against all supported Granta MI releases and a recent Granta MI development build.

  • All PyGranta meta-package and idiomatic package released release branches can be patched and used for releases if required. Changes in behavior from subsequent Granta MI development do not affect existing release branches.

Note

If a production test VM has been decommissioned, it should be removed from release branch CI if a patch release is required.