Databricks has MLflow in version 1.12 published. The platform for managing machine learning (ML) projects primarily brings some improvements in the interaction with PyTorch in the current release. The developers presented the new features at the first PyTorch Developer Day, which took place as a virtual event on November 12. November as a virtual event.
MLflow voluntarily keeps log
MLflow 1.12 introduces a universal autlogging feature independent of the extended PyTorch integration: The method mlflow.autolog() shall automatically log all relevant model properties such as parameters, metrics, and artifacts. Until now, this required calling the respective methods for the individual entities.
For interaction with PyTorch the current release brings the API mlflow.pytorch.autologous to automatically log metrics, parameters and models with. MLflow can now also create automated logs from PyTorch lightning models. The performance optimized framework for model training has been released in October in version 1.0 was released.
MLflow can now also load and manage TorchScript. TorchScript can be used to create models that can be serialized and do not require Python dependencies. Just-in-time compiler (JIT) translation can be nudged from MLflow, as can loading and logging, as the following code from the Databricks blog shows:
# Any PyTorch nn.modules or pl.LightningModule model = Net() scripted_model = torch.jit.script(model) … mlflow.pytorch.log_model(scripted_model, "scripted_model") model_uri = mlflow.get_artifact_uri("scripted_model") loaded_model = mlflow.pytorch.load_model(model_uri) …
Freshly served and cleverly explained
For the distribution of applications version 1.12 includes a plug-in for integration with TorchServe. Facebook introduced the deployment library in conjunction with Amazon Web Services earlier this year. The mlflow-torchserve plug-in allows models trained from MLflow pipelines to be deployed with TorchServe.
The plug-in transfers previously trained models to production via TorchServe.
Another new feature beyond the PyTorch integration is the mlflow method.shap.log_explanation to log model declarations according to SHapley Additive exPlanations (SHAP), an approach inspired by game theory to explain the output of ML models.
Other new features in MLflow 1.12 can be found on the Databricks blog. A complete list of additions and bugfixes can be found in the release notes on GitHub. Developers can install the software via the Python package manager PyPI with the command pip install mlflow. The source code is stored in the GitHub repository.