AI label detection for point clouds

AI label detection can be used in label detection of point clouds in eShare after it has been enabled and configured by a system administrator. See AI support.

AI label detection can be used in Point Clouds and Textured Meshes configuration view.

Detecting labels

Do the following:

  1. Navigate to the project to edit, and then click Project Admin in the main menu. The project administration view opens.

  2. Click Point Clouds and Textured Meshes. The Point Clouds and Textured Meshes configuration view opens.

  3. In Point Clouds and Textured Meshes section, select the correct folder, and then all the point cloud files you want to include.

  4. Select Detect labels with AI. If the detection cannot be started, hovering over the button will display the reason.

    A dialog with the following prompt Start detection for files? Are you sure you want to start detection for files? This will send your selected files to the selected AI service. is shown.

  5. Select Start Detection. Detection begins.

  6. To view the progress, select Detection logs. The progress for each individual point cloud is shown.

  7. To stop detection during the process, select Stop detection.

  8. After the detection has completed, the following outcomes are shown depending on the results:

    • If new labels are found, click on New labels to analyze or AI Results to view them. The buttons are disabled until detection is completed in all folders.

    • No new labels found: the view will remain the same as it was before detection was started.

    • If an error occurs while detecting labels, click on the error indicator to view the full error message.

Displaying the analysis view of labels

The analysis view allows you to filter, view, and generate Smart Points using the found labels. Analysis view consists of three panes: Filtering, Smart Point Generation Rules, and the labels themselves.

Prerequisites

  • You have completed AI label detection for point clouds and found labels.

Filtering the labels

In Filtering Rules you can filter the found labels. Specify the following:

  • Show – Select All labels to view all labels, or select Labels that haven't been dismissed or generated.

  • Fuzzy matching – Set to Yes, if you want to enable matching similar characters. For example 0 and O will be considered a match. When set to No, only identical characters are matched.

    Note: This does not apply to regex filters.

  • Case sensitive – When set to Yes, the filter differentiates between uppercase and lowercase letters. When set to No, the filter matches words regardless of capitalization.

    Note: This does not apply to regex filters.

Filtering the results

In Filtering Rules view, there are also filters for the actual results.

In Label Filtering Rules section, you can filter the results with five different preset filters:

  • Automatic Detection with Model Keys – Select the model attributes to use in matching the detection results to the selected attributes.

  • Automatic Detection with Document Object Links – Select to match the detection results with the already created, indexed links between documents and model objects attributes.

  • Automatic Detection with Document Smart Point Links – Select the Smart Point types to use in matching the detection results with already created links between documents and Smart Points.

  • Excel file – Select an Excel file to use. Select the worksheet, column, first table row, and if the document has a header row. Preview the fields from the file by selecting Preview. This filter will match the results against the data in the Excel file.

  • Filter with regexes where you can define the name of the pattern, and the pattern to be used. This will match the results against the pattern. You can also add a new regex by selecting Add new.

    You can also use the AI assistant for creating the regex by selecting Ask AI, if it has been enabled for the project. See AI assistant.

To save the current filters, select Save Configuration. To apply the filters, select Apply filters. If there are filtered results, you can select Reset results to reset to the original results.

Creating Smart Point generation rules

Smart Point Generation Rules pane allows you to select rules for creating Smart Points from the found labels. You can choose if the external ID of the Smart Point should be the label that the AI found, or the matched label from the filters, and which type of Smart Point will be created.

Prerequisites

  • You have completed AI label detection for point clouds and found labels.

Do the following:

  1. Select New labels to analyze after label detection has completed.

  2. In Smart Point Generation Rules pane, select Show Options to expand the pane, and specify the following:

    • Label to use – Select Found Label or Matched Label. If matched label with fuzzy matching is selected, and there are multiple matches to the same label, the matched label with the least changes from the found label will be used.

      Note: This does not apply to labels matched with regex.

    • Smart Point Type – Select the Smart Point type to be generated.

  3. In Label Transformation Rules, select Add new to add a new transformation rule.

  4. In Apply to Results From, select which filters this transformation should apply to, so that you can transform labels from only selected filters if desired. You can use regex named groups to modify the labels to the desired format, similar to other parts of the applications. Named groups can also be created in the regex filters found using the Label Filtering Rules and they will be carried over here.

  5. To preview the Smart Points to be generated, select Test transformations and check for duplicates, if there are transformation rules, or in all other cases select Check for duplicates. The results for the first five labels in the results view, or the selected labels if there are any, will be displayed. In Transformation Results From The Selected Labels, you can see:

    • original name of the label

    • External Id transformed from the original label based on the rules to be used

    • if there already is a Smart Point at the same coordinates with the same External Id

    • if there already is a model object with the same External Id.

    If the preview is of selected labels, you can remove the label from the selected labels by selecting Remove from selected. You can also filter the preview with the duplicate Smart Point, and duplicate External Id columns.

  6. Select Save configuration.

Viewing label detection results

Label detection results pane displays the actual results from the label detection. Depending on the results, different columns are shown. If the results are not filtered, the following will be shown:

  • Label Text – indicates the label found from the image

  • Confidence – the confidence of the AI in if the result is correct.

  • Coordinates – indicates the location of the label in the point cloud.

  • Point Cloud Source – the name of the point cloud where the label was found

  • Has Identical Smart Point – indicates if there already is a Smart Point in the same location with the found label name.

If the results are filtered, there will also be columns for

  • Matched Keys – indicates what data was matched to the label

  • Match – indicates which filter was matched.

You can filter the results by Match, Confidence, Point Cloud Source, and if the label has identical Smart Points. You can also sort the results according to Label Text, Match, or Confidence.

There also are two buttons in the Label Text column where you can edit the found label, and depending on if the label has multiple matches, view an image of the found label in the point cloud, or view all duplicate matches from different point clouds with their images on the side.

You can select the labels, and either dismiss them by selecting Dismiss, or generate Smart Points by selecting Generate Smart Points. A warning is displayed, if there are duplicate Smart Points, or items with the same External Id. If you create a large number of Smart Points at once, a confirmation dialog will be shown. A notification is displayed after the Smart Points have been successfully generated.

To delete labels, select the labels you want to delete, and select Delete.

Note: Deleting labels is permanent and cannot be undone after confirming the deletion. If you delete a point cloud the labels that were detected from, the point cloud will have the value Deleted in the point cloud source, and the image will be a warning of the point cloud being deleted.

Exporting and importing label detection configuration

Note: The exported configuration includes all point cloud, textured mesh, and label detection configurations of the project.

Do the following:

  1. Navigate to the project to edit, and then click Project Admin in the main menu. The project administration view opens.

  2. Click Point Clouds and Textured Meshes. The Point Clouds and Textured Meshes configuration view opens.

  3. To export settings, click Export All, and then copy the settings from the Export Point Cloud Options text box.

  4. To import settings, click Import, paste the required settings into the Import text box, and then click Import.