Computation backends can call __dlpack__ to obtain the underlying data in a tensor without copying the data. This allows use to use tensorflow tensors etc. without copying the data.

class onnxscript.ir.DataType(value)[source]¶

Enum for the data types of ONNX tensors, defined in onnx.TensorProto.

BFLOAT16 = 16¶

BOOL = 9¶

COMPLEX128 = 15¶

COMPLEX64 = 14¶

DOUBLE = 11¶

FLOAT = 1¶

FLOAT16 = 10¶

FLOAT8E4M3FN = 17¶

FLOAT8E4M3FNUZ = 18¶

FLOAT8E5M2 = 19¶

FLOAT8E5M2FNUZ = 20¶

INT16 = 5¶

INT32 = 6¶

INT4 = 22¶

INT64 = 7¶

INT8 = 3¶

STRING = 8¶

UINT16 = 4¶

UINT32 = 12¶

UINT4 = 21¶

UINT64 = 13¶

UINT8 = 2¶

UNDEFINED = 0¶

classmethod from_numpy(dtype: dtype) → DataType[source]¶

Returns the ONNX data type for the numpy dtype.

Raises:: TypeError – If the data type is not supported by ONNX.

property itemsize: float¶: Returns the size of the data type in bytes.

numpy() → dtype[source]¶

Returns the numpy dtype for the ONNX data type.

Raises:: TypeError – If the data type is not supported by numpy.

An immutable concrete tensor with its data store on disk.

This class uses memory mapping to avoid loading the tensor into memory, when the data type is supported by numpy. Otherwise, the tensor is loaded into memory lazily when accessed.

Calling shape does not incur IO. Checking shape before loading the tensor is recommended if IO overhead and memory usage is a concern.

To obtain an array, call numpy(). To obtain the bytes, call tobytes().

The path can be a relative path or an absolute path. Serializers should handle the path correctly to conform with the ONNX spec.

path¶: The path to the data file. This can be a relative path or an absolute path.

offset¶: The offset in bytes from the start of the file.

length¶: The length of the data in bytes.

dtype¶

The data type of the tensor.

Type:: onnxscript.ir._enums.DataType

shape¶

The shape of the tensor.

Type:: onnxscript.ir._protocols.ShapeProtocol

name¶

The name of the tensor. It must be specified.

Type:: str | None

doc_string¶

The documentation string.

Type:: str | None

metadata_props¶

The metadata properties.

Type:: MutableMapping[str, str]

doc_string: str | None¶

property dtype: DataType¶

property length: int | None¶

property meta: MetadataStore¶

The metadata store for intermediate analysis.

Write to the metadata_props if you would like the metadata to be serialized to the ONNX proto.

property metadata_props: dict[str, str]¶

name: str¶

numpy() → ndarray[source]¶

Return the tensor as a numpy array.

The data will be memory mapped into memory and will not taken up physical memory space.

property offset: int | None¶

property path: str | PathLike¶

raw: mmap.mmap | None¶

property shape: Shape¶

tobytes() → bytes[source]¶

Return the bytes of the tensor.

This will load the tensor into memory.

class onnxscript.ir.Function(domain: str, name: str, overload: str = '', *, graph: Graph, attributes: Sequence[Attr], metadata_props: dict[str, str] | None = None)[source]¶

IR functions.

Like a graph, a function can have nodes that are not topologically sorted. It is the responsibility of the user to maintain a topological order of the nodes.

Note that there is not a node attribute in the Function. The Function can be seen as a Sequence of nodes and should be used as such. For example, to obtain all nodes as a list, call list(function).

name¶

The function name.

Type:: str

domain¶

The domain this function is defined in.

Type:: str

overload¶

The overload name when the function is overloaded.

Type:: str

inputs¶

The input values of the function.

Type:: Sequence[onnxscript.ir._protocols.ValueProtocol]

attributes¶

The attributes this function defines.

Type:: OrderedDict[str, onnxscript.ir._protocols.AttributeProtocol]

outputs¶

The output values of the function.

Type:: Sequence[onnxscript.ir._protocols.ValueProtocol]

opset_imports¶

Opsets imported by the function.

Type:: MutableMapping[str, int]

doc_string¶

Documentation string.

Type:: str

metadata_props¶

Metadata that will be serialized to the ONNX file.

Type:: MutableMapping[str, str]

meta¶

Metadata store for graph transform passes.

Type:: MutableMapping[str, Any]

append(node: Node, /) → None[source]¶: Append a node to the function in O(1) time.

property attributes: OrderedDict[str, Attr]¶

property doc_string: str | None¶

property domain: str¶

extend(nodes: Iterable[Node], /) → None[source]¶: Extend the function with the given nodes in O(#new_nodes) time.

identifier() → Tuple[str, str, str][source]¶: Return the unique identifier of the function.

property inputs: list[Input]¶

insert_after(node: Node, new_nodes: Iterable[Node], /) → None[source]¶: Insert new nodes after the given node in O(#new_nodes) time.

insert_before(node: Node, new_nodes: Iterable[Node], /) → None[source]¶: Insert new nodes before the given node in O(#new_nodes) time.

property meta: MetadataStore¶

The metadata store for intermediate analysis.

Write to the metadata_props if you would like the metadata to be serialized to the ONNX proto.

property metadata_props: dict[str, str]¶

property name: str¶

property opset_imports: dict[str, int]¶

property outputs: list[Value]¶

property overload: str¶

remove(nodes: Node | Iterable[Node], /, safe: bool = False) → None[source]¶

Remove nodes from the graph in O(#num of nodes) time.

If any errors are raise, to ensure the graph is not left in an inconsistent state, the graph is not modified.

Parameters:

nodes – The node to remove.
safe –
If True, performs the following actions before removal:

1. It checks to make sure there are no users of the node that are not to be removed before removing it. 2. It checks the node does not contribute to any graph outputs. 3. It removes references to all inputs so it is no longer a user of other nodes.

Raises:

ValueError – If any node to remove does not belong to this graph.
ValueError – (When safe=True) If the node does not belong to this graph or if there are users of the node.
ValueError – (When safe=True) If the node is still being used by other nodes not to be removed.

sort() → None[source]¶: Topologically sort the nodes in the function.

class onnxscript.ir.FunctionProtocol(*args, **kwargs)[source]¶

Protocol for ONNX functions.

Like a graph, a function can have nodes that are not topologically sorted. It is the responsibility of the user to maintain a topological order of the nodes.

name¶

The function name.

Type:: str

domain¶

The domain this function is defined in.

Type:: str

overload¶

The overload name when the function is overloaded.

Type:: str

inputs¶

The input values of the function.

Type:: Sequence[onnxscript.ir._protocols.ValueProtocol]

attributes¶

The attributes this function defines.

Type:: OrderedDict[str, onnxscript.ir._protocols.AttributeProtocol]

outputs¶

The output values of the function.

Type:: Sequence[onnxscript.ir._protocols.ValueProtocol]

opset_imports¶

Opsets imported by the function.

Type:: MutableMapping[str, int]

doc_string¶

Documentation string.

Type:: str

metadata_props¶

Metadata that will be serialized to the ONNX file.

Type:: MutableMapping[str, str]

meta¶

Metadata store for graph transform passes.

Type:: MutableMapping[str, Any]

append(node: NodeProtocol, /) → None[source]¶: Append a node to the function.

attributes: OrderedDict[str, AttributeProtocol]¶

doc_string: str¶

domain: str¶

extend(nodes: Iterable[NodeProtocol], /) → None[source]¶: Extend the function with the given nodes.

identifier() → Tuple[str, str, str][source]¶: Return the unique identifier of the function.

inputs: Sequence[ValueProtocol]¶

insert_after(node: NodeProtocol, new_nodes: Iterator[NodeProtocol], /) → None[source]¶: Insert new nodes after the given node.

insert_before(node: NodeProtocol, new_nodes: Iterator[NodeProtocol], /) → None[source]¶: Insert new nodes before the given node.

meta: MutableMapping[str, Any]¶

metadata_props: MutableMapping[str, str]¶

name: str¶

opset_imports: MutableMapping[str, int]¶

outputs: Sequence[ValueProtocol]¶

overload: str¶

remove(node: NodeProtocol, /) → None[source]¶: Remove a node from the function.

sort() → None[source]¶: Topologically sort the nodes in the function.

class onnxscript.ir.Graph(inputs: Sequence[Input], outputs: Sequence[Value], *, nodes: Iterable[Node], initializers: Sequence[Value] = (), doc_string: str | None = None, opset_imports: dict[str, int] | None = None, name: str | None = None, metadata_props: dict[str, str] | None = None)[source]¶

IR Graph.

Graph represents a computation graph. In addition to the ONNX specification specified fields, it also contains a mapping of opset_imports. This allows different subgraphs to import different opsets. It is the responsibility of the deserializer to reconcile the different opsets.

The nodes are not guaranteed to be topologically sorted. But the iteration order should be deterministic across different runs. It is the responsibility of the user to maintain a topological order of the nodes.

Note that there is not a node attribute in the Graph. The Graph can be seen as a Sequence of nodes and should be used as such. For example, to obtain all nodes as a list, call list(graph).

name¶

The name of the graph.

Type:: str | None

inputs¶

The input values of the graph.

Type:: MutableSequence[onnxscript.ir._protocols.ValueProtocol]

outputs¶

The output values of the graph.

Type:: MutableSequence[onnxscript.ir._protocols.ValueProtocol]

initializers¶

The initializers in the graph.

Type:: MutableMapping[str, onnxscript.ir._protocols.ValueProtocol]

doc_string¶

Documentation string.

Type:: str

opset_imports¶

Opsets imported by the graph.

Type:: MutableMapping[str, int]

metadata_props¶

Metadata that will be serialized to the ONNX file.

Type:: MutableMapping[str, str]

meta¶

Metadata store for graph transform passes.

Type:: MutableMapping[str, Any]

append(node: Node, /) → None[source]¶

Append a node to the graph in O(1) time.

Unique names will be assigned to the node and its values if any name is None.

Parameters:: node – The node to append.
Raises:: ValueError – If the node belongs to another graph.

property doc_string: str | None¶

extend(nodes: Iterable[Node], /) → None[source]¶

Extend the graph with the given nodes in O(#new_nodes) time.

Unique names will be assigned to the node and its values if any name is None.

Parameters:: nodes – The nodes to extend the graph with.
Raises:: ValueError – If any node belongs to another graph.

property initializers: dict[str, Value]¶

property inputs: list[Input]¶

insert_after(node: Node, new_nodes: Iterable[Node] | Node, /) → None[source]¶

Insert new nodes after the given node in O(#new_nodes) time.

Unique names will be assigned to the node and its values if any name is None.

Parameters:

node – The node to insert after.
new_nodes – The new nodes to insert.

Raises:

ValueError – If any node belongs to another graph.

insert_before(node: Node, new_nodes: Iterable[Node] | Node, /) → None[source]¶

Insert new nodes before the given node in O(#new_nodes) time.

Unique names will be assigned to the node and its values if any name is None.

Parameters:

node – The node to insert before.
new_nodes – The new nodes to insert.

Raises:

ValueError – If any node belongs to another graph.

property meta: MetadataStore¶

The metadata store for intermediate analysis.

Write to the metadata_props if you would like the metadata to be serialized to the ONNX proto.

property metadata_props: dict[str, str]¶

name: str | None¶

node(index_or_name: int | str, /) → Node[source]¶

Get a node by index or name.

This is an O(n) operation. Getting nodes on the ends of the graph (0 or -1) is O(1).

Note

If you need repeated random access, consider turning it into a list with list(graph) . Or a dictionary for repeated access by name: {node.name for node in graph} .

When a name is provided and if there are multiple nodes with the same name, the first node with the name is returned.

Parameters:

index_or_name – The index or name of the node.

Returns:

The node if found.

Raises:

IndexError – If the index is out of range.
ValueError – If the node with the given name is not found.

num_nodes() → int[source]¶

Get the number of nodes in the graph in O(1) time.

Note that this method returns the number of nodes this graph directly contains. It does not count nodes in subgraphs.

This is an alias for len(graph). Use this if you prefer a more descriptive name for readability.

property opset_imports: dict[str, int]¶

property outputs: list[Value]¶

remove(nodes: Node | Iterable[Node], /, safe: bool = False) → None[source]¶

Remove nodes from the graph in O(#num of nodes to remove) time.

If any errors are raise, to ensure the graph is not left in an inconsistent state, the graph is not modified.

Parameters:

nodes – The node to remove.
safe –
If True, performs the following actions before removal:

1. It checks to make sure there are no users of the node that are not to be removed before removing it. 2. It checks the node does not contribute to any graph outputs. 3. It removes references to all inputs so it is no longer a user of other nodes.

Raises:

ValueError – If any node to remove does not belong to this graph.
ValueError – (When safe=True) If the node does not belong to this graph or if there are users of the node.
ValueError – (When safe=True) If the node is still being used by other nodes not to be removed.

sort() → None[source]¶: Topologically sort the nodes in the graph.

class onnxscript.ir.GraphProtocol(*args, **kwargs)[source]¶

Protocol for graphs.

Note that there is not a node attribute in the Graph. The Graph can be seen as a Sequence of nodes and should be used as such. For example, to obtain all nodes as a list, call list(graph).

name¶

The name of the graph.

Type:: str | None

inputs¶

The input values of the graph.

Type:: MutableSequence[onnxscript.ir._protocols.ValueProtocol]

outputs¶

The output values of the graph.

Type:: MutableSequence[onnxscript.ir._protocols.ValueProtocol]

initializers¶

The initializers in the graph.

Type:: MutableMapping[str, onnxscript.ir._protocols.ValueProtocol]

doc_string¶

Documentation string.

Type:: str

opset_imports¶

Opsets imported by the graph.

Type:: MutableMapping[str, int]

metadata_props¶

Metadata that will be serialized to the ONNX file.

Type:: MutableMapping[str, str]

meta¶

Metadata store for graph transform passes.

Type:: MutableMapping[str, Any]

append(node: NodeProtocol, /) → None[source]¶: Append a node to the graph.

doc_string: str¶

extend(nodes: Iterable[NodeProtocol], /) → None[source]¶: Extend the graph with the given nodes.

initializers: MutableMapping[str, ValueProtocol]¶

inputs: MutableSequence[ValueProtocol]¶

insert_after(node: NodeProtocol, new_nodes: Iterator[NodeProtocol], /) → None[source]¶: Insert new nodes after the given node.

insert_before(node: NodeProtocol, new_nodes: Iterator[NodeProtocol], /) → None[source]¶: Insert new nodes before the given node.

meta: MutableMapping[str, Any]¶

metadata_props: MutableMapping[str, str]¶

name: str | None¶

opset_imports: MutableMapping[str, int]¶

outputs: MutableSequence[ValueProtocol]¶

remove(node: NodeProtocol, /) → None[source]¶: Remove a node from the graph.

sort() → None[source]¶: Topologically sort the nodes in the graph.

class onnxscript.ir.GraphView(inputs: Sequence[Value], outputs: Sequence[Value], *, nodes: Iterable[Node], initializers: Sequence[TensorProtocol] = (), doc_string: str | None = None, opset_imports: dict[str, int] | None = None, name: str | None = None, metadata_props: dict[str, str] | None = None)[source]¶

A read-only view on a graph.

The GraphView is useful for analysis of a subgraph. It can be initialized with a subset of nodes from a Graph. Creating GraphView does not change the ownership of the nodes, and so it is possible to create multiple GraphViews that contain the same nodes. If the underlying nodes / connections are mutated, the mutation will be reflected in all views as well.

The graph view can be serialized to ONNX:

graph_proto = ir.serde.serialize_graph(graph_view)

It can also be used to create a model:

model = ir.Model(graph_view, ir_version=8)
model_proto = ir.serde.serialize_model(model)

The model created with a GraphView will have a fixed topology, and its graph will remain read-only as a GraphView. No copying will be done during the initialization process.

name¶: The name of the graph.

inputs¶: The input values of the graph.

outputs¶: The output values of the graph.

initializers¶: The initializers in the graph.

doc_string¶: Documentation string.

opset_imports¶: Opsets imported by the graph.

metadata_props¶: Metadata that will be serialized to the ONNX file.

meta¶: Metadata store for graph transform passes.

doc_string¶

initializers¶

inputs¶

property meta: MetadataStore¶

The metadata store for intermediate analysis.

Write to the metadata_props if you would like the metadata to be serialized to the ONNX proto.

property metadata_props: dict[str, str]¶

name¶

nodes¶

opset_imports¶

outputs¶

class onnxscript.ir.GraphViewProtocol(*args, **kwargs)[source]¶

Protocol for a read-only view on a graph.

name¶

The name of the graph.

Type:: str | None

inputs¶

The input values of the graph.

Type:: Sequence[onnxscript.ir._protocols.ValueProtocol]

outputs¶

The output values of the graph.

Type:: Sequence[onnxscript.ir._protocols.ValueProtocol]

initializers¶

The initializers in the graph.

Type:: Mapping[str, onnxscript.ir._protocols.ValueProtocol]

doc_string¶

Documentation string.

Type:: str

opset_imports¶

Opsets imported by the graph.

Type:: Mapping[str, int]

metadata_props¶

Metadata that will be serialized to the ONNX file.

Type:: MutableMapping[str, str]

meta¶

Metadata store for graph transform passes.

Type:: MutableMapping[str, Any]

doc_string: str¶

initializers: Mapping[str, ValueProtocol]¶

inputs: Sequence[ValueProtocol]¶

meta: MutableMapping[str, Any]¶

metadata_props: MutableMapping[str, str]¶

name: str | None¶

opset_imports: Mapping[str, int]¶

outputs: Sequence[ValueProtocol]¶

class onnxscript.ir.Input(name: str | None = None, shape: Shape | None = None, type: TypeProtocol | None = None, doc_string: str | None = None)[source]¶: Input of a Graph or a Function.

class onnxscript.ir.MapTypeProtocol(*args, **kwargs)[source]¶

Protocol for ONNX map types.

TODO: This protocol is not yet implemented in the ONNX IR.

key_type: Literal[DataType.STRING, DataType.INT64, DataType.INT32, DataType.INT16, DataType.INT8, DataType.UINT64, DataType.UINT32, DataType.UINT16, DataType.UINT8]¶

value_type: DataType¶

class onnxscript.ir.Model(graph: Graph, *, ir_version: int, producer_name: str | None = None, producer_version: str | None = None, domain: str | None = None, model_version: int | None = None, doc_string: str | None = None, functions: Sequence[Function] = (), meta_data_props: dict[str, str] | None = None)[source]¶

doc_string: str | None¶

domain: str | None¶

property functions: dict[Tuple[str, str, str], Function]¶

graph: Graph¶

ir_version: int¶

property meta: MetadataStore¶

The metadata store for intermediate analysis.

Write to the metadata_props if you would like the metadata to be serialized to the ONNX proto.

property metadata_props: dict[str, str]¶

model_version: int | None¶

property opset_imports: dict[str, int]¶

producer_name: str | None¶

producer_version: str | None¶

class onnxscript.ir.ModelProtocol(*args, **kwargs)[source]¶

Protocol for models.

A model is a container for a graph and metadata. It is the top-level object that represents an ONNX model.

graph¶

The graph of the model.

Type:: onnxscript.ir._protocols.GraphProtocol

ir_version¶

The version of the IR.

Type:: int

producer_name¶

The name of the producer.

Type:: str | None

producer_version¶

The version of the producer.

Type:: str | None

domain¶

The domain of the model.

Type:: str | None

model_version¶

The version of the model.

Type:: int | None

doc_string¶

Documentation string.

Type:: str | None

functions¶

The functions defined in the model.

Type:: MutableMapping[str, onnxscript.ir._protocols.FunctionProtocol]

metadata_props¶

Metadata that will be serialized to the ONNX file.

Type:: MutableMapping[str, str]

meta¶

Metadata store for graph transform passes.

Type:: MutableMapping[str, Any]

doc_string: str | None¶

domain: str | None¶

functions: MutableMapping[str, FunctionProtocol]¶

graph: GraphProtocol¶

ir_version: int¶

meta: MutableMapping[str, Any]¶

metadata_props: MutableMapping[str, str]¶

model_version: int | None¶

opset_imports: MutableMapping[str, int]¶

producer_name: str | None¶

producer_version: str | None¶

IR Node.

If the graph is provided, the node will be added to the graph. Otherwise, user is responsible to call graph.append(node) (or other mutation methods in Graph) to add the node to the graph.

After the node is initialized, it will add itself as a user of the input values.

The output values of the node are created during node initialization and are immutable. To change the output values, create a new node and replace the each of the inputs of output.uses() with the new output values by calling replace_input_with() on the using nodes of this node’s outputs.

append(nodes: Node | Iterable[Node]) → None[source]¶

Insert a node after this node in the list of nodes in the graph.

It is the same as calling graph.insert_after(self, nodes).

Example:

Before: previous_node -> self
        previous_node' -> node -> next_node'
After:  previous_node -> self -> node
        previous_node' -> next_node'

Parameters:: nodes – A node or a sequence of nodes to put after this node.

property attributes: OrderedDict[str, Attr | RefAttr]¶

display(*, page: bool | None = None) → None[source]¶

Pretty print the object.

Parameters:: page – Whether to page the output if it is too long.

doc_string: str | None¶

property domain: str¶

property graph: Graph | None¶

property inputs: Sequence[Value | None]¶

property meta: MetadataStore¶

The metadata store for intermediate analysis.

Write to the metadata_props if you would like the metadata to be serialized to the ONNX proto.

property metadata_props: dict[str, str]¶

property name: str | None¶

op_identifier() → Tuple[str, str, str][source]¶

property op_type: str¶

property outputs: Sequence[Value]¶

property overload: str¶

prepend(nodes: Node | Iterable[Node]) → None[source]¶

Insert a node before this node in the list of nodes in the graph.

It is the same as calling graph.insert_before(self, nodes).

Example:

Before: previous_node -> self
        previous_node' -> node -> next_node'
After:  previous_node -> node -> self
        previous_node' -> next_node'

Parameters:: nodes – A node or a sequence of nodes to put before this node.

replace_input_with(index: int, value: Value | None) → None[source]¶: Replace an input with a new value.

property version: int | None¶

class onnxscript.ir.NodeProtocol(*args, **kwargs)[source]¶

Protocol for nodes.

A node represents an invocation of an operation on the Value s in the computational graph.

A node can be optionally named. A name should typically be assigned when the node is added to a graph.

domain, op_type, and overload together uniquely identify the operator, and are always strings. For ONNX operators, domain and overload are both empty strings.

inputs and outputs are the input and output values of the node.

attributes are the attributes of the node. The attributes are stored in an ordered dictionary to preserve the order of the attributes. This is a deviation from the current ONNX spec where attributes are unordered, but it is helpful for tools that rely on the order of the attributes, e.g. those converting to and from Python function keyword arguments.

version is unique to the IR and is not specified in the ONNX spec. This allows the IR to represent a graph with mixed opset versions. Deserializers should decide how to reconcile the different versions within the graph. A typical graph will have a single version, declared in the Graph object and the nodes will have None as the version.

domain¶

The domain of the operator. E.g. "" for ONNX operators.

Type:: str

op_type¶

The operator name.

Type:: str

overload¶

The overload name when the node is invoking a function.

Type:: str

inputs¶

Input values.

Type:: Sequence[onnxscript.ir._protocols.ValueProtocol]

outputs¶

Output values.

Type:: Sequence[onnxscript.ir._protocols.ValueProtocol]

attributes¶

The attributes of the operator.

Type:: OrderedDict[str, onnxscript.ir._protocols.AttributeProtocol | onnxscript.ir._protocols.ReferenceAttributeProtocol]

version¶

The version of the operator.

Type:: int | None

doc_string¶

Documentation string.

Type:: str | None

metadata_props¶

Metadata that will be serialized to the ONNX file.

Type:: MutableMapping[str, str]

meta¶

Metadata store for graph transform passes.

Type:: MutableMapping[str, Any]

attributes: OrderedDict[str, AttributeProtocol | ReferenceAttributeProtocol]¶

doc_string: str | None¶

domain: str¶

inputs: Sequence[ValueProtocol]¶

meta: MutableMapping[str, Any]¶

metadata_props: MutableMapping[str, str]¶

name: str | None¶

op_type: str¶

outputs: Sequence[ValueProtocol]¶

overload: str¶

replace_input_with(index: int, value: ValueProtocol | None) → None[source]¶: Set the input at the given index to the given value, replacing the original value.

version: int | None¶

class onnxscript.ir.OptionalType(elem_type: TypeProtocol, *, denotation: str | None = None)[source]¶

A type that represents an optional element.

denotation: str | None¶

class onnxscript.ir.RefAttr(name: str, ref_attr_name: str, type: AttributeType, *, doc_string: str | None = None)[source]¶

Reference attribute.

doc_string: str | None¶

property name: str¶

property ref_attr_name: str¶

property type: AttributeType¶

class onnxscript.ir.ReferenceAttributeProtocol(*args, **kwargs)[source]¶

Protocol for a reference attribute.

A reference attribute can only appear inside the definition body of a function.

name¶

The name of the attribute.

Type:: str

ref_attr_name¶

The name of the attribute definition this attribute refers to.

Type:: str

type¶

The type of the attribute.

Type:: onnxscript.ir._enums.AttributeType

doc_string¶

Documentation string.

Type:: str | None

doc_string: str | None¶

name: str¶

ref_attr_name: str¶

type: AttributeType¶

class onnxscript.ir.SequenceType(elem_type: TypeProtocol, *, denotation: str | None = None)[source]¶

A type that represents a sequence of elements.

denotation: str | None¶

property dims: tuple[int | SymbolicDim, ...]¶

All dimensions in the shape.

This property is read-only. Use __getitem__ and __setitem__ to modify the shape or create a new shape.

get_denotation(index: int) → str | None[source]¶

Return the denotation of the dimension at the index.

Parameters:: index – The index of the dimension.
Returns:: The denotation of the dimension.

numpy() → tuple[int, ...][source]¶

rank() → int[source]¶: The rank of the shape.

set_denotation(index: int, denotation: str | None) → None[source]¶

Set the denotation of the dimension at the index.

Parameters:

index – The index of the dimension.
denotation – The denotation of the dimension.

class onnxscript.ir.ShapeProtocol(*args, **kwargs)[source]¶

Protocol for ONNX shapes.

A shape is a sequence of dimensions.

dims¶

The dimensions of the shape.

Type:: Sequence[int | onnxscript.ir._protocols.SymbolicDimProtocol]

dims: Sequence[int | SymbolicDimProtocol]¶

get_denotation(index: int) → str | None[source]¶

numpy() → Sequence[int][source]¶

rank() → int[source]¶

set_denotation(index: int, denotation: str | None) → None[source]¶

class onnxscript.ir.SparseTensorProtocol(*args, **kwargs)[source]¶

dims: Sequence[int]¶

indices: TensorProtocol¶

values: TensorProtocol¶

class onnxscript.ir.SparseTensorType(dtype: DataType, *, denotation: str | None = None)[source]¶

A type that represents a sparse tensor.

denotation: str | None¶

Multidimensional array of strings (as binary data to match the string_data field in TensorProto).

doc_string: str | None¶

property dtype: DataType¶: The data type of the tensor. Immutable.

property meta: MetadataStore¶

The metadata store for intermediate analysis.

Write to the metadata_props if you would like the metadata to be serialized to the ONNX proto.

property metadata_props: dict[str, str]¶

name: str | None¶

numpy() → npt.NDArray[np.bytes_][source]¶: Return the tensor as a numpy array.

property raw: Sequence[bytes] | npt.NDArray[np.bytes_]¶: Backing data of the tensor. Immutable.

property shape: Shape¶: The shape of the tensor. Immutable.

string_data() → Sequence[bytes][source]¶: Return the string data of the tensor.

tobytes() → bytes[source]¶: Return the tensor as a byte string conformed to the ONNX specification, in little endian.

class onnxscript.ir.SymbolicDim(value: str | None)[source]¶

property value: str | None¶

class onnxscript.ir.SymbolicDimProtocol(*args, **kwargs)[source]¶

Value of a single symbolic/dynamic dimension in a shape.

value¶

The value of the dimension.

Type:: str | None

value: str | None¶

An immutable concrete tensor.

This class is a wrapper around the raw tensor data. The raw tensor data can be a numpy array compatible object (e.g. np.ndarray, torch.Tensor) or a DLPack compatible object. The tensor is immutable and the data is not copied at initialization.

To create a tensor from a numpy array:

>>> import numpy as np
>>> array = np.array([1, 2, 3])
>>> tensor = Tensor(array)
>>> # The tensor itself can be treated as a numpy array because it implements the __array__ method
>>> np.allclose(tensor, array)
True

To get a numpy array from the tensor, call numpy(). To convert the tensor to a byte string for serialization, call tobytes().

It is recommended to check the size of the tensor first before accessing the underlying data, because accessing the data may be expensive and incur IO overhead.

Subclass this class to efficiently handle different types of tensors from different frameworks.

name¶

The name of the tensor.

Type:: str | None

shape¶

The shape of the tensor.

Type:: onnxscript.ir._protocols.ShapeProtocol

dtype¶

The data type of the elements of the tensor. It is an ir.DataType enum.

Type:: onnxscript.ir._enums.DataType

doc_string¶

Documentation string.

Type:: str | None

raw¶

The raw data behind this tensor. It can be anything.

Type:: Any

size¶: The number of elements in the tensor.

nbytes¶: The number of bytes in the tensor.

metadata_props¶

Metadata that will be serialized to the ONNX file.

Type:: MutableMapping[str, str]

meta¶

Metadata store for graph transform passes.

Type:: MutableMapping[str, Any]

doc_string: str | None¶

property dtype: DataType¶: The data type of the tensor. Immutable.

property meta: MetadataStore¶

The metadata store for intermediate analysis.

Write to the metadata_props if you would like the metadata to be serialized to the ONNX proto.

property metadata_props: dict[str, str]¶

name: str | None¶

numpy() → ndarray[source]¶

Return the tensor as a numpy array.

When the data type is not supported by numpy, the value is the bit representation of the dtype:

int8 for int4, with the sign bit extended to 8 bits.
uint8 for uint4.
uint8 for 8-bit data types like float8.
uint16 for bfloat16.

property raw: TArrayCompatible¶: Backing data of the tensor. Immutable.

property shape: Shape¶: The shape of the tensor. Immutable.

tobytes() → bytes[source]¶

Returns the value as bytes encoded in little endian.

Override this method for more efficient serialization when the raw value is not a numpy array.

class onnxscript.ir.TensorProtoTensor(proto: TensorProto)[source]¶

A tensor initialized from a tensor proto.

property doc_string: str¶

property dtype: DataType¶

property meta: MetadataStore¶

The metadata store for intermediate analysis.

Write to the metadata_props if you would like the metadata to be serialized to the ONNX proto.

property metadata_props: dict[str, str]¶

property name: str¶

numpy() → ndarray[source]¶

Return the tensor as a numpy array.

This is an improved version of onnx.numpy_helper.to_array. It first reads the data using the dtype corresponding to the tensor proto data field, then converts it to the correct dtype and shape. Special cases are bfloat16, complex and int4 where we need to reinterpret the data. Other types can simply be casted.

When the data type is not supported by numpy, the value is the bit representation of the dtype:

int8 for int4, with the sign bit extended to 8 bits.
uint8 for uint4.
uint8 for 8-bit data types like float8.
uint16 for bfloat16.

When the data type is a string, this method returns a numpy array of bytes instead of a numpy array of strings, to follow the ONNX specification.

External tensors are not supported by this class. Use onnxscript.ir.ExternalTensor instead.

Raises:: ValueError – If the data type is UNDEFINED.

property raw: TensorProto¶

property shape: Shape¶

tobytes() → bytes[source]¶

Return the tensor as a byte string conformed to the ONNX specification, in little endian.

Raises:

ValueError – If the tensor is a string tensor or an external tensor.
ValueError – If the tensor is of UNDEFINED data type.

class onnxscript.ir.TensorProtocol(*args, **kwargs)[source]¶

Concrete tensor backed by data.

The protocol does not specify how the data is stored. That data is exposed through the raw attribute for examination, but accessing raw is typically not needed.

To use the tensor as a numpy array, call numpy(). To convert the tensor to a byte string for serialization, call tobytes().

It is recommended to check the size of the tensor first before accessing the underlying data, because accessing the data may be expensive and incur IO overhead.

name¶

The name of the tensor.

Type:: str | None

shape¶

The shape of the tensor.

Type:: onnxscript.ir._protocols.ShapeProtocol

dtype¶

The data type of the elements of the tensor. It is an ir.DataType enum.

Type:: onnxscript.ir._enums.DataType

doc_string¶

Documentation string.

Type:: str | None

raw¶

The raw data behind this tensor. It can be anything.

Type:: Any

size¶: The number of elements in the tensor.

nbytes¶: The number of bytes in the tensor.

metadata_props¶

Metadata that will be serialized to the ONNX file.

Type:: MutableMapping[str, str]

meta¶

Metadata store for graph transform passes.

Type:: MutableMapping[str, Any]

doc_string: str | None¶

dtype: DataType¶

meta: MutableMapping[str, Any]¶

metadata_props: MutableMapping[str, str]¶

name: str | None¶

property nbytes: int¶

numpy() → np.ndarray[source]¶: Return the tensor as a numpy array.

raw: Any¶

shape: ShapeProtocol¶

property size: int¶

tobytes() → bytes[source]¶: Return the tensor as a byte string conformed to the ONNX specification, in little endian.

class onnxscript.ir.TensorType(dtype: DataType, *, denotation: str | None = None)[source]¶

A type that represents a tensor.

denotation: str | None¶

class onnxscript.ir.TypeAndShape(type: TypeProtocol | None, shape: Shape | None)[source]¶

Type and shape.

Useful for constructing a type proto.

shape: Shape | None¶

type: TypeProtocol | None¶

class onnxscript.ir.TypeProtocol(*args, **kwargs)[source]¶

Protocol for ONNX tensors, Sequence tensors, Optional tensors and Sparse tensors.

These three types of tensors share the same attribute “elem_type” so they are merged in the same interface. Unlike the ONNX TensorProto, shapes are not included in the type and should be stored in the Value.

denotation¶

An optional denotation can be used to denote the whole type with a standard semantic description as to what is stored inside. Refer to https://github.com/onnx/onnx/blob/main/docs/TypeDenotation.md#type-denotation-definition for pre-defined type denotations.

Type:: str | None

elem_type¶

The type of its elements for nested types like Sequence[Optional] tensors. Or the DataType if the type is not nested.

Type:: onnxscript.ir._protocols.TypeProtocol | onnxscript.ir._enums.DataType

dtype¶

The data type of the tensor or the nested tensor.

Type:: onnxscript.ir._enums.DataType

denotation: str | None¶

dtype: DataType¶

elem_type: TypeProtocol | DataType¶

IR Value.

A value is a named entity that can be used to represent an input or output of a graph, a function, or a node. The information it stores generalizes over ValueInfoProto in the ONNX specification.

A Value is always not owned or owned by exactly one node. When the value is not owned, it must be an input of a graph or a function. producer and index are None.

When the value is owned by a node, it is an output of the node. The node that produces the value can be accessed with producer(). The index of the output of the node that produces the value can be accessed with index().

To find all the nodes that use this value as an input, call uses().

To check if the value is an output of a graph, call is_graph_output().

name¶

The name of the value. A value is always named when it is part of a graph.

Type:: str

shape¶

The shape of the value.

Type:: onnxscript.ir._protocols.ShapeProtocol | None

type¶

The type of the value.

Type:: onnxscript.ir._protocols.TypeProtocol | None

metadata_props¶

Metadata.

Type:: MutableMapping[str, str]

property const_value: TensorProtocol | None¶

A concrete value.

The value can be backed by different raw data types, such as numpy arrays. The only guarantee is that it conforms TensorProtocol.

doc_string: str | None¶

property dtype: DataType | None¶: The data type of the tensor.

index() → int | None[source]¶: The index of the output of the defining node.

is_graph_output() → bool[source]¶: Whether the value is an output of a graph.

property meta: MetadataStore¶

The metadata store for intermediate analysis.

Write to the metadata_props if you would like the metadata to be serialized to the ONNX proto.

property metadata_props: dict[str, str]¶

property name: str | None¶

producer() → Node | None[source]¶

The node that produces this value.

When producer is None, the value does not belong to a node, and is typically a graph input or an initializer.

property shape: Shape | None¶

property type: TypeProtocol | None¶

The type of the tensor.

Example types can be TensorType, SparseTensorType, SequenceType, OptionalType. To obtain the data type of the tensor, use type.dtype or conveniently dtype.

uses() → Collection[tuple[Node, int]][source]¶

Return a set of uses of the value.

The set contains tuples of (Node, index) where the index is the index of the input of the node. For example, if node.inputs[1] == value, then the use is (node, 1).

class onnxscript.ir.ValueProtocol(*args, **kwargs)[source]¶

Protocol for values.

A value is a named entity that can be used to represent an input or output of a graph, a function, or a node. The information it stores generalizes over ValueInfoProto in the ONNX specification.

A Value is always not owned or owned by exactly one node. When the value is not owned, it must be an input of a graph or a function. producer and index are None.

To find all the nodes that use this value as an input, call uses().

To check if the value is an output of a graph, call is_graph_output().

name¶

The name of the value. A value is always named when it is part of a graph.

Type:: str

shape¶

The shape of the value.

Type:: onnxscript.ir._protocols.ShapeProtocol | None

type¶

The type of the value.

Type:: onnxscript.ir._protocols.TypeProtocol | None

metadata_props¶

Metadata that will be serialized to the ONNX file.

Type:: MutableMapping[str, str]

meta¶

Metadata store for graph transform passes.

Type:: MutableMapping[str, Any]

doc_string¶

Documentation string.

Type:: str | None

const_value¶

The constant tensor is the value constant.

Type:: onnxscript.ir._protocols.TensorProtocol | None

const_value: TensorProtocol | None¶

doc_string: str | None¶

index() → int | None[source]¶: The index of the output of the node that produces this value.

is_graph_output() → bool[source]¶: Whether this value is an output of a graph.

meta: MutableMapping[str, Any]¶

metadata_props: MutableMapping[str, str]¶

name: str¶

producer() → NodeProtocol | None[source]¶: The node that produces this value.

shape: ShapeProtocol | None¶

type: TypeProtocol | None¶

uses() → Collection[tuple[NodeProtocol, int]][source]¶: The set of (node, input_index) with node being those that use this value as an input.

onnxscript.ir.from_proto(proto: ModelProto | GraphProto | NodeProto | TensorProto | AttributeProto | ValueInfoProto | TypeProto) → Any[source]¶: Deserialize an ONNX proto message to an IR object.

onnxscript.ir.to_proto(ir_object: ModelProtocol | GraphProtocol | NodeProtocol | ValueProtocol | AttributeProtocol | ReferenceAttributeProtocol | TensorProtocol | TypeProto | GraphViewProtocol) → Any[source]¶: Serialize an IR object to a proto.

onnxscript.ir¶

onnxscript.ir ¶