ONNX vs. Core ML: Choosing the Best Approach for Model Conversion in 2024

In the rapidly evolving world of machine learning, deploying models efficiently across different platforms has become a top priority. Two major options for model conversion and deployment are ONNX (Open Neural Network Exchange) and Core ML, each tailored to specific needs and ecosystems. This article dives deep into their differences, strengths, and when to use one over the other, while also exploring the intricacies of converting complex models like Stable Diffusion to Core ML.

What is ONNX?

ONNX is an open-source format designed to enable interoperability of machine learning models across various frameworks. It’s particularly useful for deploying models on diverse platforms, such as Windows, Linux, and Android.

Advantages of ONNX

1. Framework Agnosticism:

• Models trained in frameworks like PyTorch, TensorFlow, or MXNet can be exported to ONNX and deployed elsewhere.

1. Optimization:

• ONNX Runtime offers accelerated inference through hardware optimization and techniques like quantization (float16, int8).

1. Cross-Platform Flexibility:

• ONNX is compatible with multiple operating systems and hardware platforms, making it a versatile choice for developers.

Limitations of ONNX

1. Static Computation Graphs:

• ONNX primarily supports static graphs, which can limit its use for dynamic or complex models.

1. Complexity in Conversion:

• Advanced models like diffusion-based architectures often face compatibility challenges during ONNX conversion.

How to Use ONNX

Here’s an example workflow to convert a PyTorch model to ONNX and then use ONNX Runtime for inference:

import torch
import torch.onnx
import onnxruntime as ort

# Define a simple PyTorch model
class SimpleModel(torch.nn.Module):
    def __init__(self):
        super(SimpleModel, self).__init__()
        self.fc = torch.nn.Linear(10, 5)

    def forward(self, x):
        return self.fc(x)

# Instantiate and save the model
model = SimpleModel()
x = torch.randn(1, 10)

# Export the model to ONNX
torch.onnx.export(
    model,               # Model
    x,                   # Input tensor
    "model.onnx",        # ONNX file path
    input_names=['input'],
    output_names=['output'],
    dynamic_axes={'input': {0: 'batch_size'}, 'output': {0: 'batch_size'}}
)

# Run inference with ONNX Runtime
ort_session = ort.InferenceSession("model.onnx")
inputs = {"input": x.numpy()}
outputs = ort_session.run(None, inputs)
print(outputs)

What is Core ML?

Core ML is Apple’s proprietary framework designed for deploying machine learning models on iOS, macOS, watchOS, and tvOS devices. It is highly optimized for Apple’s ecosystem, including the Neural Engine and GPUs on Apple Silicon.

Advantages of Core ML

1. Apple Silicon Optimization:

• Core ML leverages Apple’s hardware for efficient, on-device inference.

1. Ease of Integration:

• Models in .mlmodel format integrate seamlessly with Xcode projects.

1. Support for Advanced Models:

• Recent updates enable stateful models and advanced architectures, making Core ML ideal for tasks like sequential data processing.

1. Built-in Compression:

• Core ML supports quantization to reduce model size while maintaining performance.

Limitations of Core ML

1. Platform Specificity:

• Core ML models are primarily limited to Apple’s ecosystem, making cross-platform deployment difficult.

1. Conversion Challenges for Certain Models:

• While powerful, converting some advanced models like Stable Diffusion requires specialized tools.

How to Use Core ML

Here’s an example workflow to convert a PyTorch model to Core ML using coremltools:

import torch
import coremltools as ct

# Define a simple PyTorch model
class SimpleModel(torch.nn.Module):
    def __init__(self):
        super(SimpleModel, self).__init__()
        self.fc = torch.nn.Linear(10, 5)

    def forward(self, x):
        return self.fc(x)

# Instantiate and save the model
model = SimpleModel()
x = torch.randn(1, 10)

# Convert to Core ML
traced_model = torch.jit.trace(model, x)
mlmodel = ct.convert(
    traced_model, 
    inputs=[ct.TensorType(shape=x.shape)],
)

# Save the Core ML model
mlmodel.save("model.mlmodel")

Converting Stable Diffusion Models to Core ML

Stable Diffusion, a popular generative model for creating images from text, involves intricate operations and dynamic sampling pipelines. Converting such models to ONNX can be challenging due to ONNX’s static graph limitations. Instead, direct conversion to Core ML is a more effective approach.

Core ML Conversion with MochiDiffusion

The MochiDiffusion GitHub project provides a step-by-step guide for converting Stable Diffusion models to Core ML.

Steps:

1. Prepare the Model:

• Obtain Stable Diffusion models in .ckpt or .safetensors formats.

1. Convert to Diffusers Format:

• Use Hugging Face’s Diffusers library to structure the model components: from diffusers import StableDiffusionPipeline # Load and save in Diffusers format pipeline = StableDiffusionPipeline.from_pretrained("model_directory") pipeline.save_pretrained("diffusers_model")

1. Convert to Core ML:

• Use the coreml-stable-diffusion package:
  bash python -m coreml_stable_diffusion.convert \ --model-path diffusers_model \ --output-path model.mlmodel

1. Optimize for Apple Silicon:

• Enable float16 precision for smaller model sizes and faster inference.

Updated Comparison: ONNX vs. Core ML (2024)

When to Use ONNX or Core ML

Choose ONNX if:

• You need cross-platform compatibility.
• Your model architecture is relatively simple and fits within ONNX’s static graph capabilities.
• You want the flexibility to deploy on diverse hardware and operating systems.

Choose Core ML if:

• Your app is exclusively targeting Apple’s ecosystem.
• You’re working with complex models requiring dynamic behavior or advanced optimizations.
• You want to leverage Apple’s Neural Engine for efficient on-device inference.

Conclusion

In 2024, the choice between ONNX and Core ML depends on your project’s goals, target platforms, and model complexity. While ONNX excels in cross-platform deployment, Core ML stands out for Apple-specific projects requiring advanced performance and optimization. For complex models like Stable Diffusion, direct Core ML conversion remains the best option due to its support for dynamic operations and efficient integration within Apple’s ecosystem.

By staying informed about the latest tools and techniques, developers can make strategic decisions to ensure efficient and effective model deployment. Whether you’re optimizing for cross-platform reach or diving deep into Apple’s ecosystem, the right approach can make all the difference.

For more insights and tutorials, check out MochiDiffusion and Apple’s Core ML documentation.

Please leave this field empty

don't miss our new posts. Subscribe for updates

We don’t spam! Read our privacy policy for more info.

Check your inbox or spam folder to confirm your subscription.