Dr. rer. nat.

Cheng, Chih-Hong

Filters

3
2 1
Reset filters

Settings
Now showing 1 - 3 of 3
  • Publication
    Logically Sound Arguments for the Effectiveness of ML Safety Measures
    ( 2022-09-07)
    Cheng, Chih-Hong
    ;
    Schuster, Tobias
    ;
    Burton, Simon
    We investigate the issues of achieving sufficient rigor in the arguments for the safety of machine learning functions. By considering the known weaknesses of DNN-based 2D bounding box detection algorithms, we sharpen the metric of imprecise pedestrian localization by associating it with the safety goal. The sharpening leads to introducing a conservative post-processor after the standard non-max-suppression as a counter-measure. We then propose a semi-formal assurance case for arguing the effectiveness of the post-processor, which is further translated into formal proof obligations for demonstrating the soundness of the arguments. Applying theorem proving not only discovers the need to introduce missing claims and mathematical concepts but also reveals the limitation of Dempster-Shafer’s rules used in semi-formal argumentation.
  • Publication
    Selected Challenges in ML Safety for Railway
    ( 2022-09)
    Cheng, Chih-Hong
    Neural networks (NN) have been introduced in safety-critical applications from autonomous driving to train inspection. I argue that to close the demo-to-product gap, we need scientifically-rooted engineering methods that can efficiently improve the quality of NN. In particular, I consider a structural approach (via GSN) to argue the quality of neural networks with NN-specific dependability metrics. A systematic analysis considering the quality of data collection, training, testing, and operation allows us to identify many unsolved research questions: (1) Solve the denominator/edge case problem with synthetic data, with quantifiable argumentation (2) Reach the performance target by combining classical methods and data-based methods in vision (3) Decide the threshold (for OoD or any kind) based on the risk appetite (societally accepted risk).
  • Publication
    Formally Compensating Performance Limitations for Imprecise 2D Object Detection
    ( 2022-08-25)
    Schuster, Tobias
    ;
    Seferis, Emmanouil
    ;
    Burton, Simon
    ;
    Cheng, Chih-Hong
    In this paper, we consider the imperfection within machine learning-based 2D object detection and its impact on safety. We address a special sub-type of performance limitations related to the misalignment of bounding-box predictions to the ground truth: the prediction bounding box cannot be perfectly aligned with the ground truth. We formally prove the minimum required bounding box enlargement factor to cover the ground truth. We then demonstrate that this factor can be mathematically adjusted to a smaller value, provided that the motion planner uses a fixed-length buffer in making its decisions. Finally, observing the difference between an empirically measured enlargement factor and our formally derived worst-case enlargement factor offers an interesting connection between quantitative evidence (demonstrated by statistics) and qualitative evidence (demonstrated by worst-case analysis) when arguing safety-relevant properties of machine learning functions.