Prefect integrations for interacting with Hex.
Hex is a powerful platform for collaborative data science and analytics. For information getting started with Hex, check out Hex's quickstart guide.
The tasks within this collection were created by a code generator using Hex's OpenAPI spec.
Hex's REST API documentation can be found here.
Requires an installation of Python 3.7+.
We recommend using a Python virtual environment manager such as pipenv, conda or virtualenv.
These tasks are designed to work with Prefect 2.0. For more information about how to use Prefect, please refer to the Prefect documentation.
pip install prefect-hex
A list of available blocks in
prefect-hex and their setup instructions can be found here.
Gather and store authentication
- Create new token on https://app.hex.tech/ Settings page:
- Store token on https://app.prefect.cloud/ Blocks page:
- Copy project ID from browser URL (in red):
Write and run a flow
Trigger a Hex project run and wait for completion
from prefect import flow from prefect_hex import HexCredentials from prefect_hex.project import trigger_project_run_and_wait_for_completion @flow def trigger_project_run_and_wait_for_completion_flow(project_id: str): hex_credentials = HexCredentials.load("hex-token") project_metadata = trigger_project_run_and_wait_for_completion( project_id=project_id, hex_credentials=hex_credentials ) return project_metadata trigger_project_run_and_wait_for_completion_flow( project_id="012345c6-b67c-1234-1b2c-66e4ad07b9f3" )
Run project, get status, cancel run, and get list of projects
from prefect import flow from prefect_hex import HexCredentials from prefect_hex.project import ( get_project_runs, run_project, get_run_status, cancel_run, ) @flow def example_hex_flow(): # load stored credentials hex_credentials = HexCredentials.load("hex-token") # run project project_id='5a8591dd-4039-49df-9202-96385ba3eff8', project_run = run_project(project_id=project_id, hex_credentials=hex_credentials) # get status run_id = project_run.run_id project_run_status = get_run_status( project_id=project_id, run_id=run_id, hex_credentials=hex_credentials ) print(project_run_status.run_url) # cancel run if needed cancel_run(project_id=project_id, run_id=run_id, hex_credentials=hex_credentials) # get list of project runs project_runs = get_project_runs( project_id=project_id, hex_credentials=hex_credentials ) return project_runs example_hex_flow()
For more tips on how to use tasks and flows in a Collection, check out Using Collections!
Below is a list of Blocks available for registration in
To register blocks in this module to view and edit them on Prefect Cloud:
prefect block register -m prefect_hex
loadmethod on Blocks, you must already have a block document saved through code or saved through the UI.
To load the HexCredentials:
from prefect import flow from prefect_hex.credentials import HexCredentials @flow def my_flow(): my_block = HexCredentials.load("MY_BLOCK_NAME") my_flow()
- Create Observable and Reproducible Notebooks with Hex by Khuyen Tran
If you encounter any bugs while using
prefect-hex, feel free to open an issue in the prefect-hex repository.
If you have any questions or issues while using
prefect-hex, you can find help in either the Prefect Discourse forum or the Prefect Slack community.
Feel free to star or watch
prefect-hex for updates too!
-git clone https://github.com/PrefectHQ/prefect-hex.git
If you'd like to help contribute to fix an issue or add a feature toprefect-hex`, please propose changes through a pull request from a fork of the repository.
-cd prefect-hex/ Here are the steps:
- Fork the repository
- Clone the forked repository
- Install the repository and its dependencies:
pip install -e ".[dev]"
- Make desired changes
- Add tests
- Insert an entry to CHANGELOG.md
pre-committo perform quality checks prior to commit:
git push, and create a pull request install