Development and release processes#

Definitions#

  • PyGranta meta-package: A package which depends on the idiomatic packages and auto-generated packages. Includes documentation describing the PyGranta ecosystem, including this document.

  • Idiomatic package: A hand-written package. Typically depends on one auto-generated package and openapi-common. There are currently three idiomatic packages in PyGranta:

    • ansys-grantami-bomanalytics

    • ansys-grantami-recordlists

    • ansys-grantami-jobqueue

  • Auto-generated package: A package generated by the codegen template. There are currently two auto-generated packages in PyGranta:

    • ansys-grantami-bomanalytics-openapi

    • ansys-grantami-serverapi-openapi

  • Codegen template: The template used to generate the auto-generated packages.

  • Production test VM: Virtual machine used for CI integration tests and for executing notebooks as part of the documentation build. This machine always runs the Granta MI version required to build the most recently released release branch for the PyGranta meta-package and all idiomatic packages.

  • Development test VM: Virtual machine used for CI integration tests and for executing notebooks as part of the documentation build. This machine is upgraded as required to support CI on idiomatic package main branches during development.

Development and release timeline#

The timeline is split into three distinct phases:

  1. Phase 1: Development. There is active development upstream on closed-source Granta MI components, the codegen template, and on the open source idiomatic packages and auto-generated packages. Effort is made to ensure CI is passing on all main branches, but from time to time CI may fail. The development test VM is upgraded as required to support CI passing on main branches.

  2. Phase 2: Hardening. Bugs are fixed and minor improvements are made to idiomatic packages, auto-generated packages, and the codegen template. During this phase the codegen template and auto-generated packages are finalized.

  3. Phase 3: Pre-release. The upstream closed-source development has completed. The PyGranta meta-package and idiomatic packages are manually tested and finalized.

Phase 1: Development#

During development, the following processes happen continuously:

  1. Incorporate changes to upstream API definitions into auto-generated package repositories.

  2. Check that code generation is successful by monitoring CI in the auto-generated package repositories.

  3. If code generation is not successful, resolve by either fixing the codegen template, fixing the auto-generated package configuration, or raise the issue with the internal Ansys development team.

If idiomatic package development does not depend on upstream API development, then there is no dependency between the idiomatic package and the auto-generated package and work can proceed in parallel.

If significant idiomatic package development is being done that depends on the current state of an auto-generated package, you should update the main branch of the corresponding idiomatic package to depend on the auto-generated package main directly.

During this phase, the development test VM may be upgraded if required to support new upstream Granta MI capabilities.

Phase 2: Hardening#

Phase 2a: Initial dev releases#

  1. Update the development test VM with the latest Granta MI software and databases. See the ansys-grantami-bomanalytics repository for details about preparing the Restricted Substances test database.

  2. Publish auto-generated packages to the internal PyPI mirror with .dev version numbers.

  3. Update idiomatic package main branches to depend on the auto-generated package .dev releases. If the main branch has not been updated to depend on the auto-generated package main branch during development, then there may be integration issues discovered at this point - these should be resolved in the PR before merging. The PR typically includes:

    • Updates to Python version support to align with changes to the auto-generated package.

    • Updates to the README.rst to reflect any changes in the minimum required Granta MI version.

    • Changes at the interface with the auto-generated package if either the codegen template or API definition has changed.

    • Changes to expected test results. This is typically relevant for ansys-grantami-bomanalytics which includes acceptance-style tests.

  4. Publish idiomatic packages to the internal PyPI mirror with .dev version numbers.

    Note

    At this point release branches should not be created. .dev releases should be published to the internal PyPI mirror from the main branch.

    Note

    These .dev releases can be used for internal testing of closed-source packages that have runtime dependencies on PyGranta packages.

  5. Continue to review any changes to API definitions via auto-generated packages throughout the hardening phase. If there are any issues with the API definitions, these must be escalated with high priority.

Phase 2b: Documentation review#

Review the documentation for all idiomatic packages and the PyGranta meta-package:

  • Significant new documentation is reviewed by the appropriate reviews for linguistic and technical accuracy.

  • References to dependencies are updated, including:

    • Server software versions

    • Required Python versions

    • Compatible third-party library versions

  • References to Ansys documentation of other packages/products are checked to see if they need to be updated to the previous version of that documentation.

    Note

    The Ansys documentation for the current in-development release is published after the PyGranta packages are released. The documentation for the previous release should be used to ensure links in published PyGranta documentation can always be resolved.

Phase 2c: Release candidates#

Towards the end of the hardening phase, once the API definitions are determined to be stable, the auto-generated packages are released:

  1. Create release branches for auto-generated packages and publish as release candidates to public PyPI.

  2. Create Pull Requests in idiomatic package repositories to update main to depend on the auto-generated package release candidate releases. Ensure CI passes for idiomatic packages.

  3. Publish auto-generated packages as stable releases to public PyPI.

  4. Create Pull Requests in idiomatic package repositories to update main to depend on the auto-generated package stable releases. Ensure CI passes for idiomatic packages.

  5. Create release branches for idiomatic packages and publish as release candidates to public PyPI.

  6. Update the PyGranta meta-package main branch to depend on the public idiomatic packages release candidates and auto-generated packages stable releases.

    Note

    At this point release branches should be created for idiomatic packages. However, during hardening these release branches still run CI against the development test VM.

    Note

    Idiomatic package release candidates must depend on stable release versions of auto-generated packages only. They must not depend on pre-release versions of auto-generated packages.

    Note

    When performing pre-releases of packages with towncrier, the Ansys action creates a Pull Request to update CHANGELOG.md on main, and commits the CHANGELOG.md changes directly to the release branch. The commit to the release branch should be reverted, and the Pull Request to merge the changes to main should be closed and not merged.

The stable auto-generated packages and release candidate idiomatic packages can now be used for the final release of internal packages. It is recommended to make any closed-source idiomatic package release candidate dependencies flexible enough such that they match subsequent stable releases of those idiomatic packages.

At the end of this phase, the PyGranta meta-package depends on release candidates for the idiomatic packages and releases of the auto-generated packages. This guarantees that there exists a consistent set of dependencies across all idiomatic packages and auto-generated packages. Any issues at this stage should be resolved by addressing any dependency conflicts in the idiomatic packages and publishing new release candidates to public PyPI as necessary.

Checklist#

  • The development test VM is running an up-to-date development version of Granta MI.

  • CI for all idiomatic packages is passing.

  • Documentation for all idiomatic packages and the PyGranta meta-package has been reviewed to verify significant new content, changes in dependencies (for example Python version support, minimum Granta MI version), and references to other Ansys documentation sources.

  • release branches have been created for auto-generated packages and idiomatic packages.

  • Stable auto-generated packages are available on PyPI.

  • Internal packages with runtime dependencies on auto-generated packages depend on the stable PyPI releases.

  • Idiomatic packages depend on stable releases of auto-generated packages only.

  • Release candidate idiomatic packages are available on PyPI.

  • Internal packages with runtime dependencies on idiomatic packages depend on the release candidate PyPI releases.

  • The PyGranta meta-package main branch depends on idiomatic packages release candidates and auto-generated packages stable releases only.

Phase 3: Pre-release#

  1. Update the production test VM to run the Granta MI release validated during the hardening phase.

  2. Update the latest idiomatic package release branches created during the hardening phase to run CI against the production test VM.

  3. Manually test the PyGranta meta-package against the Granta MI release candidate by installing the main branch directly using pip.

Note

If there have been significant changes to the PyGranta meta-package, a release candidate may optionally be published to PyPI. Examples of significant changes include re-organization of the documentation and changes to the CI process, both of which are not checked as part of the manual test required in this phase.

Testing should occur 4 weeks following Phase 2. Any issues raised during testing should be immediately triaged and fixed or deferred.

Checklist for completing this phase:

  • The production test VM is running the release version of Granta MI.

  • CI for all idiomatic packages is passing.

  • Manual testing has been completed against the PyGranta meta-package and all idiomatic packages.

  • The GitHub PyGranta Development project contains no must-deliver features or bug fixes still outstanding for the active release.

Phase 4: Release#

The PyAnsys meta-package is typically released a week before Granta MI is available to customers. As a result, the stable versions of the packages should be released at least a week before Granta MI is available to customers:

  1. Defer any issues still associated with the active release in the GitHub PyGranta Development to a future release.

  2. Publish idiomatic packages as stable releases to public PyPI.

  3. Update the PyGranta meta-package main branch to depend on the public idiomatic packages stable releases.

  4. Create (or update if a release candidate was published in Phase 3) a release branch for the PyGranta meta-package.

  5. Publish the PyGranta meta-package as a stable release to public PyPI.

  6. Create a Pull Request in the PyAnsys meta-package repository to move the PyGranta meta-package dependency to the new stable release.

  7. If any new idiomatic package releases are compatible with the previous release of Granta MI and PyGranta, including the minimum supported Python version, create a patch release of the previous PyGranta meta-package release which includes the new idiomatic package releases.

Checklist for completing this phase:

  • The active release in the GitHub PyGranta Development project is empty, with all deferred tickets re-assigned to a future release.

  • Stable releases of all idiomatic packages and the PyGranta meta-package are available on PyPI.

  • The PyAnsys meta-package depends on the PyGranta meta-package stable release.

  • A patch release has been made for the previous PyGranta meta-package if there is at least one backwards- compatible idiomatic package released during this development cycle.

Addenda#

Test VM management#

At all times the production test VM virtual machine should be running the latest validated Granta MI version. This is generally the most recent released version, but during the pre-release phase the version of Granta MI installed on the production test VM is more recent than the latest version available to customers. The name and URL of the production test VM is stored in the AZURE_VM_NAME and TEST_SERVER_URL secrets respectively.

The development test VM runs whichever version of Granta MI is required to support CI on idiomatic package main branches during development. The name and URL of the development test VM is stored in the AZURE_VM_NAME_NEXT and TEST_SERVER_URL_NEXT secrets respectively.

The production test VM and development test VM can be distinguished by their desktop backgrounds and text files in the hard drive root.

This approach achieves two goals:

  • CI is generally passing on all idiomatic package main branches at any point in development and release.

  • The most recent PyGranta meta-package and all idiomatic package released release branches can still be built up to the pre-release phase to support patch releases if required.

On this page