Some Notes on Ballistic Systems Interoperability Study via X3P

In 2025-2026 as a part of EMPACT Operational Action 1.3 a study was carried out with the aim to evaluate interoperability capabilities between several commercial Ballistic Identification Systems including Evofinder, BalScan and IBIS.

Each system scanned the same dataset, which included 200 cartridge cased fired from 100 different guns, resulting in 100 known matched pairs of samples. The samples were scanned by each BIS, and the resulting data were converted by each manufacturer from its proprietary format into X3P files.

The goal of this document is to provide an objective summary of the reported search effectiveness and to identify and rebut one-sided, clearly incorrect conclusions resulting from the manipulation of data presentation.

It focuses on comparison of search effectiveness reported by Evofinder and IBIS manufacturers, while using their own proprietary search algorithms.

The data used in this document are stated in the presentations made in Nanterre (France) during the EMPACT meeting in June 2026:

1. Interoperability Performance analysis Using IBIS Tools. A first step towards the Europol Ballistic Hub,

2. EMPACT Dataset Analysis Report. Inter-systems X3P Data Exchange. Evofinder Processed.

The first two columns of the table below describe the sources of compared objects (column 1) that are correlated against background data set (column 2). Mentioned combination of data sets then are correlated against each other by using the Evofinder and IBIS search algorithms for breech face (columns 3 and 4) and firing pin (columns 5 and 6) groups of traces. Search effectiveness shown in the table is calculated as the percentage of known matching samples found in first place in the corresponding hit list.

NOTE. The samples were scanned by the law enforcement organizations participating in the project. It appears that the guideline on correct sample orientation was not followed consistently during data acquisition with the Evofinder system. For some matching samples, the difference in orientation angle exceeds the 45-degrees margin and, in some cases, reached up to 180 degrees. At the same time, samples acquired with the BalScan and IBIS systems are orientated very carefully, with angular deviation close to zero. This introduced a disadvantage in the search effectiveness results measured for the Evofinder data. The effect is particularly significant for firing-pin searches result and may reduce the measured effectiveness by up to 10%.

Analysis of the table data shows:

1. Search algorithms from both observed systems demonstrate more or less close results for the breech face group of traces. At that, Evofinder shows slightly higher search effectiveness for almost all combination of data except of IBIS to IBIS search (77% vs 80.5%, see the table line 5).

That can be explained by different quality of images scanned on mentioned systems and corresponding search algorithms adjusted to concrete image quality. The Evofinder system images contain more detailed marks (high spatial frequencies).

Fig. 1. Images of the same sample acquired by the Evofinder and IBIS systems.

That is also proved by close results for the Evofinder to BalScan search (84.5% and 82%, see the table lines 1 and 2) as both these systems produce more similar images.

Despite the significant difference in image quality, the observed search effectiveness for the Evofinder to Evofinder search (84.5%) and the Ibis to Ibis search (77%) does not differ so dramatically. That can be explained with relatively small size of background database. The available low spacial frequency information on IBIS images is sufficient to find most samples in the first place of a list containing 199 candidates. However, due to low dynamic range of such images, search effectiveness is expected to decrease significantly as the size of the background database increases. The images obtained by the Evofinder system contain significantly more high spacial frequency information, which makes the decrease of search effectiveness less pronounced as the size of the background database increases.

The impact of different spacial frequencies on search effectiveness was analyzed in a study presented at the annual ENFSI meeting in 2019: “The influence of ballistic traces spatial frequencies on automatic search effectiveness”. The graph presented in that study is shown below. It clearly demonstrates that search effectiveness for samples with spatial frequencies limited to 128 micrometers decreases several times faster than for the same dataset containing images with spacial frequencies limited to 32 or 16 micrometers (see Fig. 2).

Fig 2. Impact of different spatial frequencies on the search effectiveness.

2. More significant difference of the search effectiveness is observed for firing pin group of traces. It corresponds to the image quality in both systems (see highlighted area in Fig. 1). In this case results of search algorithms drops down to 7.5% while using IBIS correlation algorithms for Evofinder to IBIS search (see line 3).

Usage of more effective Evofinder algorithms give ~2.5 times better result for Evofinder to IBIS search (20.5%, see line 3).

Mentioned difference in image quality is well illustrated on slide 8 of the presentation [1] (see Fig. 3).

Fig 3. Images of the same sample acquired by the BalScan, Evofinder, and IBIS systems.

Finally, that is important to emphasize that the objective of the OA1.3. EMPACT project is to evaluate the X3P format compatibility, but results were presented by some companies as X3P performance tests that was not the purpose of the study. Before carrying out proper and objective comparison of search effectiveness of different systems, it is mandatory to return to the roots of the project:

1. Formalize requirements for the key factors of effective data exchange i.e. quality of images, their resolution and lossless conversion of the proprietary data to X3P.

2. Develop and define requirements for automated transferring specific ballistic data (firearms and ammunition attributes, regions of interest) between BIS. At the moment the X3P format is just a container for a plain “3D image”.