Competitions Rules (registration, submission, presentation)

Participants should start preparing a descriptive text for their final submission. Indeed, top-ranked teams will be invited to present their works at the MetroXRAINE 2024 conference during a Poster Session. Further details can be found here.

No later than 07/07/2024 31/07/2024 (extended) participants must register at this registration form where they must specify the team members (first name, last name, affiliation, email contact) and the name of their team. It will not be allowed to unify teams after the beginning of the challenge even if they belong to the same university. Training data will be released on 08/07/2024 19/07/2024 (extended), while test data will be released on 01/08/2024 05/08/2024 (extended).

The technical documentation that participants must submit no later than 11/08/2024 31/08/2024 (extended) must be written in English and must contain all the details of the proposed approach (for example, if you use a deep neural network algorithm then you must describe the architecture used, the parameters, etc.) and the results obtained, as well as the comparison with the methods reported here. Please, submit your solution using this form.

For this purpose, templates will be provided to compile the report containing the details of the proposed method and the csv file containing the results. All templates will be available here.

Criteria of judging a submission

• Classification Accuracy: Precision in correctly distinguishing between the two classes is crucial. Accuracy will be used as a starting point to assess the overall model performance.
• Precision and Recall: Precision indicates the proportion of true positive predictions among all positive predictions. Recall measures the proportion of true positive predictions among all actual positive instances. A good balance between precision and recall is desirable, but their importance may vary depending on the context of the problem.
• F1 Score: The F1 score is the harmonic mean of precision and recall. This metric provides a balance between the two and can be particularly useful in cases where minimizing both false positives and false negatives is important.

Dataset to be used

The dataset to be used consists of a new dataset created and provided by the challenge organizers. The dataset consists of images obtained by plotting the backscatter values measured by the ceilometer, a special measuring instrument used in meteorology, capable of detecting the height of a cloud base by emitting a modulated beam of light directed towards the sky. The height of the clouds is calculated by measuring the time of flight of the emitted light beam.

The ceilometer takes measurements every 15 seconds, quantifying the concentration of particles in the atmosphere. By taking advantage of the reflection, it is possible to determine the cloud layer coverage.

Once raw data were collected, the variables of interest were normalized using a normalization factor of the lidar-based ceilometer. The variable containing the backscatter coefficient is also used in addition to the time variable. All the variables are put together to be used to generate backscatter profiles. Specifically, we plot the time on the x-axis and the height of the measured particles (contained in the backscatter coefficient) on the y-axis. We generated a backscatter profile for each day of measurements and each of these were further divided for every hour. The produced backscatter profiles were labeled with the support of a Weather Research and Forecasting (WRF) Model.

Training Set Now Available!

You can download the training set here. Our baseline solution achieves the following values:

• Classification Accuracy: 89,5745.
• Precision: 0,8982.
• Recall: 0,9584.
• F1 Score: 0,9273.

Test Set Now Available!

The link to download the test set was sent to registered teams by email. You can download it here. For any needs, please contact us.