Using filtering in query strings

Calls requesting data from endpoints may include a query string that filters the data to make read operations quicker and to decrease the size of the response.

The CADMATIC Web API supports the following filtering methods: query filters, reply filters, and pagination. Query filters and reply filters are supported on all GET endpoints that return a collection of items, while pagination is only supported on selected GET endpoints.

Query filters

You can add query filters to the query string to enable the API call to target only items that have a given property, and optionally, a given value in that property.

You can include multiple query filters or use both query filters and reply filters in the same call. If the call includes multiple query filters, the results from the individual queries are combined using a logical AND operator. Combining multiple query filters with OR is not directly supported, but you can perform multiple GET requests and combine the results on the client side.

Syntax of query filters | Rules for applying query filters

Syntax of query filters

Query filters can be constructed using the following syntax:

<neg><jsonfield>|<jsondictfield>|attributes(<string>)<operator><value>

where

<neg> is an optional exclamation mark (!) that causes each condition to be negated and evaluated separately. Negation is most useful when the filter does not include an optional operator–value condition.
<jsonfield>|<jsondictfield>|attributes(<string>) specifies the targeted field in the response. The query checks whether the field exists and evaluates for false if it does not.
- <jsonfield> specifies the targeted JSON property using dot notation: object.properties.objectType="Drawing"
- <jsondictfield> specifies the targeted JSON property using bracket notation: [<string>]. You can also mix dictionary like access and the <jsonfield> path: ["object"].properties["objectType"]="Drawing"
- attributes(<string>) specifies a COS attribute tag as "<string>". If the string itself contains double-quote (") or backslash (\) characters, they must be escaped with the backslash (\) character.

<operator><value> defines an optional condition of the targeted field. If the operator is specified, then also the value must be given.

<operator> can be one of the following:

=	equal to
!=	not equal to
<=	less than or equal to
<	less than
>=	greater than or equal to
>	greater than
R	regular expression
in	included in the given set
notin	not included in the given set

Note: The use of comparison operator requires the value to be a string, a number, or a Boolean. Other types of values cause the filter to be ignored.

<value> can be one of the following, depending on the data type of the JSON field:

string	A quoted string.
number	A quoted string, or just the number without the quotes.
Boolean	"true" or "false"
list	A comma-separated list of items inside square brackets, where each item is a quoted string or a number: [<item1>, <item2>, …, <itemn>]

Rules for applying query filters

If the filter does not have an operator–value condition, it only checks if the object has the given field.

?query=properties.ModificationTime

The filter finds objects that have the 'ModificationTime' property.

If the filter has an operator–value condition, it checks if the field exists and if its value satisfies the given condition.

?query=properties.ModificationTime>1645034215

The filter finds objects where 'ModificationTime' is greater than the given value.

If the filter is negated with a leading exclamation mark, it evaluates each part of the filter expression separately.

?query=!properties.ModificationTime

The filter finds objects that do not have the 'ModificationTime' property.
?query=!properties.ModificationTime>1645034215

The filter finds (a) objects that do not have the 'ModificationTime' property and (b) objects where 'ModificationTime' is not greater than the given value.

If the filter contains an element that is an array, the rest of the filter's path is used to evaluate every item in the array. If at least one of the items matches the filter, the object is considered a match.

?query=connectionPoints.data.tg1

The filter finds objects where any of the connection points has the key 'tg1' in its data.
?query=!connectionPoints.data.tg1

The filter finds objects where none of the connection points has the key 'tg1' in its data.
?query=!connectionPoints["data"]["a.b"]

The filter uses bracket notation to find objects where none of the connection points has the key 'a.b' in its data. The filter cannot use dot notation because the key contains a dot, which is a special character.

If the filter tries to compare the value of an element that is an array, the filter is ignored.

?query=connectionPoints=5

The filter is ignored. An array cannot equal a number or a string.

Reply filters

You can add reply filters to the query string to enable the API call to remove data fields that are not needed in the response, as well as to put fields back into it.

You can include multiple reply filters or use both reply filters and query filters in the same call.

Syntax of reply filters | Rules for applying reply filters

Syntax of reply filters

Reply filters can be constructed using the following syntax:

where each <replyfilter> can define the elements <exclusionSpecifier><jsonfield>|attributes(<string>) as follows.

<exclusionSpecifier> can be either '-' or '+', and it can target all the fields or just a specific field.
- -<fieldName> removes the specified field from the response.
- - removes all the fields from the response. If none are put back, the object remains as an empty object.
- +<fieldName> (or just <fieldName>) adds the specified field back into the response.
- + adds all the fields back into the response.
<jsonfield> specifies the targeted JSON field using dot notation.

Because attributes are not hard-coded special fields, but rather runtime values, they cannot be specified by the <jsonfield> syntax.
attributes(<string>) specifies attributes in the targeted field. <string> is a double-quoted string that contains the tag name of some COS attribute. If the string itself contains double-quote (") characters, they must be escaped with the backslash (\) character.

Rules for applying reply filters

If the filter contains an exclusion specifier, the filter can remove fields from the response or put fields back into it.

?reply=-properties

The filter removes all property fields from the response.
?reply=+properties.foobar

The filter puts the 'foobar' property back into the response.

If the call includes more than one reply filter, the filters are applied in the order that they are given.

?reply=-,foobar

If an object in the response has these properties:

{
  "foo" : "bar"
  "bar" : 123
  "foobar": [1, 2, 3]
}

the first filter removes every property from the object and the second filter puts the 'foobar' property back into it:

{
  "foobar": [1, 2, 3]
}

(Alternatively, the same result could have been achieved by ?reply=-foo,-bar.)

If the path of the reply filter includes a field that is an array, the filter is applied to all the items in the array.

?reply=-foobar.a

If an object in the response has this property:

{
"foobar": [{ "a": 1 }, {"b": 2}, {"a": 1}]
}

the filter removes all the 'a' fields from the property:

{
"foobar": [{}, {"b": 2}, {}]
}

You can add pagination to the query string to start reading the response from a specific object (numeric offset, key-based offset), and to read only a subset of the objects from the response (limiting).

Numeric offset

In numeric offset, the 'offset' parameter specifies the zero-based index of the first object to be read (that is, how many objects to skip before starting to read).

GET api/Project.pms/catalogParts?offset=10

Key-based offset

In key-based offset, the 'after' parameter specifies the COS OID of the object after which to start the reading.

GET api/Project.pms/catalogParts?after=GkbG.pzZHiACC6j4RQYmE0

Tip: Key-based offset provides better performance than numeric offset.

Limiting

In limiting, the 'limit' parameter defines the maximum number of objects to be read from the set.

GET api/Project.pms/catalogParts?limit=50

Combining an offset and limiting

You can use an offset parameter and limiting in the same query string. In the following example, the first query string specifies a limit of 50 objects, which is used to get the OID of the last item in that subset ('GkbG.pzZHiACC6j4RQYmE0' in our example), and then another query string reads the next 50 objects. This process can be repeated until all items have been read; this is when the returned list is either empty or contains fewer items than the specified limit.

GET api/Project.pms/catalogParts/?limit=50

GET api/Project.pms/catalogParts/?after=GkbG.pzZHiACC6j4RQYmE0&limit=50

Combining pagination and query filters

You can use pagination together with Query filters, but note the following:

The 'offset' and 'limit' parameters are applied to the filtered list, so their effect depends on the query filter.
The 'after' parameter is applied before filtering, and it works even if the object that the parameter refers to is filtered out.

Page consistency

Reading the same page again keeps the same order of items, but the list of items might change.

If an object has been removed from COS, re-reading the page removes the object from the page.
If a new object has been added into COS, re-reading the page adds the object to the page, in accordance with the existing order of items.

Therefore, clients must not assume that re-reading a page will always return exactly the same results.

Filtering examples

The following examples demonstrate the use of query filters and reply filters.

Example 1: Read catalog parts with any author

You can use a query filter to find entities that have a certain property, regardless of the values of that property.

Task: Get all catalog parts that have the 'Author' attribute (tag 'U00' in our examples) from the project database ('Project.pms' in our examples).

Query string:

?query=attributes("U00")

Example query:

GET api/Project.pms/catalogParts/?query=attributes("U00")

Example 2: Read catalog parts with specific author

You can use a query filter to find entities that have a certain property set to a certain value.

Task: Get all catalog parts that have the 'Author' attribute set to 'Cadmatic User' from the project database.

Query string:

?query=attributes("U00")="Cadmatic User"

Example query:

GET api/Project.pms/catalogParts/?query=attributes("U00")="Cadmatic User"

Example 3: Read catalog parts with no author

You can use a query filter to find entities that do not have a certain property.

Task: Get all catalog parts that do not have the 'Author' attribute from the project database.

Query string:

?query=!attributes("U00")

Example query:

GET api/Project.pms/catalogParts/?query=!attributes("U00")

Example 4: Read changes made to model objects

You can use a logical AND to combine multiple query strings.

Task: Get all changes that have been made to model objects in the project database.

Query string:

?query=objectType="ModelObject"&query=transactionType="Modified"

Example query:

GET api/Project.pms/changes/?query=objectType="ModelObject"&query=transactionType="Modified"

Example 5: Exclude properties and references from catalog parts

You can use a reply filter to remove unnecessary fields from the JSON response.

Task: Get all catalog parts from the project database. Save bandwidth by excluding all of their properties and references to model objects from the response.

Query string:

?reply=-properties,-referencedObjects

Example query:

GET api/Project.pms/catalogParts/?reply=-properties,-referencedObjects

Example 6: Exclude properties, while keeping some fields

You can use a reply filter to get only some sub-properties from the properties structure of the JSON response. You can do this by first removing all the properties and then putting back only the necessary properties.

Task: Include the creation time of the object, but exclude all other properties from the response.

Query string:

?reply=-properties,properties.creationTime

Example 7: Combine query and reply filters

You can use query filters and reply filters in the same query string.

Task: Get all catalog parts created by a specific author within the last two days. Times must be given as a Unix timestamp; for the purposes of this example, we assume that timestamp '1673524566' represents some time from two days ago. Exclude all properties, including the 'Author' attribute, from the response (the author is the same for every part).

First, let's create the reply filter string:

?reply=-properties,-attributes("U00")

Then, let's create query filter strings for specific values of 'Author' and 'CreationTime':

?query=attributes(“U00”)="Cadmatic User"

?query=properties.CreationTime>=1673524566

Finally, let's combine the reply filter and the query filters into one query string:

?reply=-properties,-attributes("U00")&query=attributes("U00")="Cadmatic User"?query=properties.CreationTime>=1673524566

So, this query returns all catalog parts that were created in the last two days and where 'Author' is set to 'Cadmatic User'. The response itself will not include the 'Author' attribute or any of the properties since the Web API was explicitly told to remove them from the response.

Example 8: Use set operations

You can use the 'in' and 'notin' operators in query filters to match a property against a set of values, instead of making a separate query for each possible value.

Task: Get the objects whose COS object type is or is not 'Model Object' or 'Catalog Part'.

Query strings:

?query=properties.ObjectType in ["Model Object", "Catalog Part"]

?query=properties.ObjectType notin ["Model Object", "Catalog Part"]

Example 9: Use regular expressions

You can use regular expressions in query filters to find items that have specific property values.

Task: Get the Systems where the 'Name' attribute (tag '.dD') contains the substring 'water' or 'Water'.

Query string:

?query=Attributes(".dD")=R"water|Water"

Example 10: Use dictionary like access

You can use bracket notation when the JSON field name includes characters that are not compatible with the standard dot notation. For example, COS attribute tags can contain dots.

Task: Get the entities whose creation time is at least a given point in time.

Query string, bracket notation:

?query=properties["CreationTime"] >= 1673524566

Query string, dot notation:

?query=properties.CreationTime >= 1673524566

Using filtering in query strings

Query filters

Syntax of query filters

Rules for applying query filters

Reply filters

Syntax of reply filters

Rules for applying reply filters

Pagination

Filtering examples

Example 1: Read catalog parts with any author

Example 2: Read catalog parts with specific author

Example 3: Read catalog parts with no author

Example 4: Read changes made to model objects

Example 5: Exclude properties and references from catalog parts

Example 6: Exclude properties, while keeping some fields

Example 7: Combine query and reply filters

Example 8: Use set operations

Example 9: Use regular expressions

Example 10: Use dictionary like access