Settings

CLASS - Settings(

• training_settings: TrainingSettings =  |factory|
• global_quantization_settings: BaseQuantizationSettings =  |factory|
• global_lora_settings: Optional[LoraSettings] =  None
• deployment_settings: Optional[BaseDeploymentSettings] =  None
• distributed_training_settings: Optional[DistributedTrainingSettings] =  None
• layer_settings: Optional[Dict[str, LayerSettings]] =  |factory|

)

This Dataclass holds all the information necessary to run CLIKA Compression.

Class Variables

• training_settings (TrainingSettings) - Training Settings
• global_quantization_settings (BaseQuantizationSettings) - Quantization Settings that are applied Globally
• global_lora_settings (Optional[LoraSettings]) - Lora settings that are applied to all eligible layers, e.g. Linear, Conv, ConvTranspose, Embedding https://arxiv.org/abs/2106.09685
• deployment_settings (Optional[BaseDeploymentSettings]) - Deployment Settings that are applied Globally. Must be set.
• distributed_training_settings (Optional[DistributedTrainingSettings]) - Settings that relate to Distributed Training for compression
• layer_settings (Optional[Dict[str, LayerSettings]]) - Settings that are applied for specific Layers.

Methods

save

save(

• path: Union[str, Path]]

) ‑> None

Saves the setting configuration to a YAML file.

load_from_path

load_from_path(

• path: Union[str, Path]]

) ‑> Optional[Settings]

Loads the setting configuration from a YAML file.

initialize_from_dict

initialize_from_dict(

• settings: dict

) ‑> Settings

Initializing a Settings object from a dict.

get_layer_settings

get_layer_settings(

• name: str

) ‑> Optional[LayerSettings]

Get the LayerSettings for a given Name.

Parameters:

• name (str) - Name of Layer

Returns:

• Optional[LayerSettings]: The Layer Settings for the given Layer with the name argument

set_layer_settings

set_layer_settings(

• name: str
• settings: LayerSettings

) ‑> None

Set LayerSettings for a given Layer name.

Parameters:

• name (str) - Name of Layer
• settings (LayerSettings) - Settings of the Layer.

set_quantization_parameters_for_layer

set_quantization_parameters_for_layer(

• name: str
• skip_quantization: Optional[bool] =  None
• skip_quantization_downstream: Optional[bool] =  None
• skip_quantization_until: Union[str, Tuple[str], List[str], None] =  None
• weights_num_bits: Optional[int] =  None
• activations_num_bits: Optional[int] =  None

) ‑> None

Sets quantization settings directly for a given layer with name. This is basically a shortcut instead of putting Quantization Settings in LayerSettings and then setting it for a specific layer.

Parameters:

• name (str) - Name of the Layer, can be derived either from the pytorch model standard layer name, or from the block name on the model architecture CLIKA Graph that is generated after training as "model_init.svg" inside the outputs folder
• skip_quantization (Optional[bool]) - Whether or not to skip quantization for this layer
• skip_quantization_downstream (Optional[bool]) - Whether or not to skip quantization from this node and downstream in the graph
• skip_quantization_until (Optional[Union[str, Tuple[str], List[str]]]) - Whether to skip quantization from the current node to the specified node or nodes given by name
• activations_num_bits (Optional[int]) - Number of bits to use for the activations of the layer
• weights_num_bits (Optional[int]) - Number of bits to use for the weights of the layer

set_quantization_settings_for_layer

set_quantization_settings_for_layer(

• name: str
• settings: BaseQuantizationSettings

) ‑> None

Set Quantization Settings for a specific Layer. This is basically a shortcut instead of putting Quantization Settings in LayerSettings and then setting it for a specific layer.

Parameters:

• name (str) - name of the Layer, can be derived either from the pytorch model standard layer name, or from the block name on the model architecture CLIKA Graph that is generated after training as "model_init.svg" inside the outputs folder
• settings (BaseQuantizationSettings) - The Quantization Settings for a specific Layer.