Skip to main content
Version: 25.5.1

FAQ

Why is tracing the model necessary?​

PyTorch models by-nature are represented as a "dynamic graph", but in order for frameworks to execute the model, they must know how the model is constructed. A benefit of purely static graphs is that more optimizations can be done once the structure of the model is clear.

My model fails to parse​

If the model is parsed successfully but ACE reports unsupported operators, contact us and we will gladly implement the missing operator.

If torch.dynamo fails to parse the model, a model modification may be required. In most cases the fix is straightforward. Overall, torch.dynamo works well for almost every model.

Why is tracing_inputs argument to clika_compile is necessary?​

By default, if tracing_inputs is not provided to clika_compile, it will just use the calibration_inputs for the tracing. (See Compile API)

The torch.dynamo module, while great, has some down-sides. After an initial call to torch.compile(), we provide model inputs in order for torch.dynamo to specialize the model for different input shapes. One potential pitfall is that torch.dynamo does not specialize based on input shapes that have dimensionality of 1. For example, assume we provide the following inputs to the model and see the output explanation:

  • (1, 3, 640, 640) - assumes 1, 3, 640, 640 are static
  • (1, 3, 530, 228) - specializes on 530, 228 (they changed)
  • (4, 3, 640, 640) - specializes on 4 (batch size dimension changed)

"Specializes" means that torch.dynamo will assume the tensor in question is not a constant shape (e.g. tensor.shape[0] to grab batch_size dimension).

In general, torch.dynamo assumes the shapes are static until they change to something that is different from 1.

In case the calibration_inputs already include the shape specializations, there is no need to provide tracing_inputs.

ACE Precompiling phase takes long time​

Easiest solution would be first to update to the latest supported PyTorch version. ACE Precompiling phase depends on torch.dynamo. The improvements being done to torch.dynamo with every new release of PyTorch are usually significant.

If that does not help, it may be that your model has very complicated shape inferencing to do. Such an example used to be maxvit from torchvision package. We're happy to assist in any case.