Databricks Wheel Package Management

Python Wheel package is a standardized format for distributing the required files for Python package installation. The Lazsa Platform provides predefined templates to execute complex data processing workflows leveraging the power of Databricks to process data efficiently. These templates are python programs that define the data flow.

Support for a new feature, enhancement to an existing functionality, or the requirement of a new template to support newer technologies are some of the scenarios in which the wheel package needs to be updated with the latest libraries and files. The Lazsa Platform provides a UI option to update wheel packages. You have the option of updating the wheel package version to the latest available one, or changing the existing wheel package version to any available version, depending on your use case.

We recommend that you update the wheel package through the Lazsa Platform and refrain from performing a manual update. This ensures that the wheel package versions in the Lazsa Platform and in the Databricks cluster are in sync. If you perform a manual update that results in a discrepancy in the wheel package versions in the Databricks cluster and in the Lazsa Platform, then follow these steps:

  1. Downgrade the wheel package to an earlier version using the Change option.

  2. Update the wheel package to the latest version using the Update option.

Updating or changing the wheel package version

Whenever a wheel package upgrade is available, it is indicated on the home page of Data Pipeline Studio as a blinking dot above the ellipsis (…).

Wheel pakage upgrade indicator

  1. Click the ellipsis (...) on the home page of Data Pipeline Studio and then click Manage Wheel Package - Upgrade Available.

  2. On the Manage Packages side drawer, the following options are seen:

  • Update to Latest - This option lets you upgrade the wheel package to the latest available version.

  • Change - This option lets you change the wheel package version to any of the last 3 versions of wheel package. You can select the required wheel package version and click Change.

    For example, if the current version is 1.0.10 and you click Change, you will see the following versions in the dropdown list:

    • 1.0.07

    • 1.0.08

    • 1.0.09

Related Topics Link IconRecommended Topics What's next? More Options or Features of Data Pipeline Studio