coleleavitt/opaque-lattice
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
feat/ake-wrapper
opaque-lattice/papers_txt/Quality-assessment-for-software-data-validation-in-aut_2026_Computer-Standar.txt
Cole Leavitt dfa968ec7d initial
2026-01-06 12:49:26 -07:00

1162 lines
127 KiB
Plaintext
Raw Permalink Blame History

This file contains invisible Unicode characters
This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
Computer Standards & Interfaces 97 (2026) 104110
Contents lists available at ScienceDirect
Computer Standards & Interfaces
journal homepage: www.elsevier.com/locate/csi
Quality assessment for software data validation in automotive industry: A
systematic literature review
Gilmar Pagoto a,* , Luiz Eduardo Galvão Martins a , Jefferson Seide Molléri b
a
UNIFESP – Federal University of São Paulo, São José dos Campos, SP 12.247-014, Brazil
b
Kristiania University of Applied Sciences, Kirkegata 17, Oslo 0167, Norway
A R T I C L E I N F O A B S T R A C T
Keywords: Context: The complexity of automotive systems continues to grow, making software quality assessment crucial for
Software quality assessment vehicle performance, safety, and cybersecurity.
Software quality characteristics Objectives: This study explores Quality Assessment (QA) in this context, focusing on its key characteristics,
Automotive software validation
practical implications, and expected deliverables.
Automotive software safety
Automotive software performance
Method: We performed a systematic literature review (SLR) by selecting 60 studies from digital libraries.
Automotive software cybersecurity Results: This SLR highlighted essential QA characteristics that should be incorporated into a software validation
phase. Our insights encourage the exploration of advanced techniques, such as Artificial Intelligence (AI), and
Machine Learning (ML), to support safety-critical software quality assessments in the automotive domain.
Conclusion: The QA of software data validation requires a holistic approach that combines safety, security, and
customer expectations, aligned with industry standards, requirements, and specifications. The relevance of AI
and ML in managing complex technologies is evidenced, and the traditional real-world validation dependencies
bring risks for safety-critical systems validation.
1. Introduction restrict coverage and prevent them from offering a fully comprehensive
and efficient approach to safety and cybersecurity.
Based on the IEEE 1012–2016 standard [1], software data validation In addition, aspects related to driver experience pose unique chal­
involves an evaluation process, as part of a system or component, con­ lenges, as reference criteria are typically established by trained pro­
ducted during or after the development process to verify compliance fessionals rather than end-users, whose specific use cases may not be
with specified requirements. This standard establishes the guidelines for fully addressed. To fulfil these already-known application variations,
Verification and Validation (V&V) that provide an objective assessment minimise the uncovered use cases, and align with customer quality
of products throughout the life cycle. In this work, the target product perception, more effort and resources are required [3].
level is the automotive software. Furthermore, ensuring quality in this Moreover, certain quality aspects are not directly visible to the end-
evaluation process is crucial for vehicle quality, particularly in terms of user but are still important for ensuring the product’s safety throughout
safety and security. This is a complex task, and this complexity increases its lifecycle. Poor software quality can lead to accidents, potentially
with the integration of emerging technologies. Advances such as resulting in material damage, financial loss, or even fatalities. ISO
autonomous driving, Advanced Driver Assistance Systems (ADAS), standards provide support and serve as foundational recommendations
connectivity systems, electric propulsion vehicles, battery electric ve­ for software data validation [4–7], contributing to the process robust­
hicles (BEV), and hybrid electric vehicles (HEV) are expanding the de­ ness. In summary, the extensive scope and growing complexity of QA for
mands on software validation. Additionally, technologies like Artificial software data validation motivated this research.
Intelligence (AI), and Machine Learning (ML) play a key role in The primary goals of this study are to identify and categorise soft­
enhancing safety system performance and improving the overall ware quality approaches used in the automotive industry according to
customer experience. their characteristics, applications, and focal areas. This study also seeks
Traditional Quality Assessment (QA) tools, as Failure Mode and Ef­ to enhance automotive professionals’ and researchers’ understanding of
fects Analysis (FMEA) [2], are no longer sufficient. Their limitations the critical role of quality in software data validation by outlining
* Corresponding author.
E-mail address: gilmar.pagoto@unifesp.br (G. Pagoto).
https://doi.org/10.1016/j.csi.2025.104110
Received 27 November 2024; Received in revised form 30 August 2025; Accepted 2 December 2025
Available online 4 December 2025
0920-5489/© 2025 Elsevier B.V. All rights are reserved, including those for text and data mining, AI training, and similar technologies.
G. Pagoto et al. Computer Standards & Interfaces 97 (2026) 104110
practical implications, expected deliverables, and the direct effects of 2.2. Automotive software data validation challenges
QA efforts.
The contributions of this Systematic Literature Review (SLR) include Selecting and ensuring software data validation in automotive sys­
the identification of the most applied interventions for QA in software tems is a complex task. Meeting end-user expectations often involves
data validation, their relevant characteristics, the impacts of applied subjective factors that are directly or indirectly influenced by software.
interventions, and state-of-the-art trends that suggest future directions Another key challenge lies in linking objective software quality metrics,
for research. as determined by QA methods, with the subjective evaluations of
This paper is organised as follows: Section 2 presents the background experienced professionals [3].
and related work. Section 3 describes the adopted research methodol­ Many Software Reliability Growth Models (SRGMs) rely heavily on
ogy. The results and analysis based on research questions are presented large datasets of failure information and often make unrealistic as­
in Section 4. In Section 5, we presented our conclusions. sumptions about software quality estimation [11]. While SRGMs can
measure quality by tracking remaining faults, failure rates, and overall
2. Background and related work software reliability, they struggle to capture customer perspectives and
insights [12].
This section reviews relevant international standards, discusses the In Automotive Driver Assistance Systems (ADAS), validating soft­
limitations of traditional QA tools, and offers insights into the current ware quality in real-world conditions is costly and time-consuming. A
state of research in software quality validation. malfunction during this phase can lead to serious accidents. To mitigate
such risks, vehicle-in-the-loop (ViL) testing can be employed. The main
2.1. Software quality standards in the automotive industry challenge with ViL is ensuring that the virtual data inputs to the vehi­
cle’s sensors are consistent with real-world inputs [13].
In software engineering (SE), the ISO/IEC 25,000 series [8], known Traditional tools for QA, such as Fault Tree Analysis (FTA), Failure
as SQuaRE (Software Quality Requirements and Evaluation), provides Mode and Effect Analysis (FMEA), and current software quality stan­
guidelines for managing, modelling, measuring, specifying, and evalu­ dards, often fall short when it comes to safety-critical systems, particu­
ating software quality. It outlines a framework for developing quality larly machine learning-based systems. This occurs mainly because of the
measures, including those focused on quality in use (ISO/IEC 25,022), high complexity of data-driven systems without a widely accepted en­
system and software product quality (ISO/IEC 25,023), and data quality gineering solution to deal with that. As a result, more refined, adapted,
(ISO/IEC 25,024). Specifically, ISO/IEC 25,012 addresses software data and specialised QA approaches are required [14].
validation, defining 15 quality characteristics grouped into inherent and This work aims to provide a comprehensive picture of the most
system-dependent perspectives. Furthermore, the IEEE 1012 standard critical software data characteristics observed in recent years, discuss
for System, Software, and Hardware Verification and Validation (V&V) their contributions to the aforementioned challenges during software
provides recommendations for evaluating the compliance of develop­ validation phases, and provide insights for handling the increasing
ment products with their intended use requirements throughout the life complexity in the software-defined vehicles domain.
cycle.
In the automotive software domain, ISO 21,434 [9] focuses on 2.3. Related work
cybersecurity assurance, addressing software defects, incorrect imple­
mentations, and vulnerabilities in security data, especially for embedded Feitosa et al. [15] conducted a systematic mapping study (SMS) with
software controlling vehicle functions like stability control. Functional the main goal of “analyse existing software engineering literature for the
Safety (FuSa), outlined in ISO 26,262 [10], addresses potential hazards purpose of characterising the state of the art concerning approaches (e.
in electric and electronic safety-related systems, aligning with the g., processes, methods, and tools) for designing critical embedded sys­
system-dependent data characteristics of ISO/IEC 25,012 to support tems (CES) from the point of view of researchers and practitioners in the
robust software data validation. context of software-intensive systems engineering”. The study identified
Table 1 shows a link between standards and their practical applica­ seven critical quality characteristics (dependability, fault-tolerance,
tion examples. performance, reliability, safety, security, and timeliness). Its most rele­
vant outcome is the identification and characterization of the created
and/or used approaches to design CES.
In 2005, Bhansali [16] proposed an optimised subset for a universal
Table 1
software standard focused on safety. This paper identifies the minimum
Practical contributions of standards for software data validation.
safety-critical quality characteristics set for multi-domain applications,
Standard Short description Use-case contributions such as: “commercial, military and space aviation; medical diagnostic
ISO/ Comprehensive Structured Evaluation of software data in- and therapeutic instruments; automotive and transportation systems;
IEC25000 guidelines for in-use software use characteristics on inherent industrial process control and robotics; nuclear power plants and
Series data quality. and/or system levels (e.g.
weapons control; commercial appliances and ride electronics”. The
Efficiency, Completeness,
Accessibility, Consistency, etc.). proposed benefits provided by a common safety standard are lower cost
ISO21434 Cybersecurity in electrical and Software integrity analysis and quality software improvement, achieved through common pro­
electronic systems, including against potential external cesses and tools.
software. attacks through functional Rana et al. [17] presented a study of software prediction methods
software tests, vulnerability
scanning, fuzz testing, and/or
focused on defect detection. The authors presented an overview of 8
penetration testing. prediction models and described the applicability of each method
ISO26262 Functional safety requirements Checking of software data for overall software lifecycles. The integrity of software data monitoring is a
Series management, design, safety-related systems (e.g., critical factor not only for the software’s performance but also for safety
implementation, verification, autonomous systems) submitted
and security. This study pointed out that post-release software moni­
validation, and confirmation to malfunctioning behaviour
measures conditions. toring was limited in the past due to a lack of software monitoring
IEEE1012 Support for Verification and Automotive function during the in-use phase, or insufficient skills and the data retrieval
Validation (V&V), including specification analysis based on capabilities.
software. vehicle requirements before Myllyaho et al. [18] presented an SLR on reported validation
software construction phase.
methods for AI-specific systems. The study classifies the applied
2
G. Pagoto et al. Computer Standards & Interfaces 97 (2026) 104110
methods into trial, simulation, model-centred validation, and expert 2.4. Definitions
opinion, and offers a detailed description of their application in realistic
settings. Gezici and Tarhan [19] conducted an SLR to investigate the Key terms are defined in Appendix A.
state-of-the-art practices on software quality (SQ) approaches for
AI-based systems. They identified quality characteristics, models, re­ 3. Methodology
ported challenges by selected studies, and suggested future work on
definition/specification, design/evaluation, and process/socio-technical The SLR was conducted followed the guidelines by Kitchenham and
categories. Furthermore, Ali et al. [20] conducted a SMS on the AI Charters [22] and Biolchini et al. [23]. Fig. 1 shows the steps of the SLR
systems, software, and components. This study identified no work on AI process, which are grouped into three main phases: planning, con­
on software component quality models, a potential gap in that knowl­ ducting, and reporting.
edge area.
Krichen [21] explored various formal methods and validation tech­
niques focused on automotive systems security. The study provides a 3.1. Planning
comprehensive overview of the state-of-the-art for systems security
validation, including the automotive lifecycle, requirements engineer­ As its objective and motivation (step1), this SLR aims to provide the
ing, design, implementation, and test phases. The challenges identified existing QA processes, methodologies, frameworks, models, or tech­
are scalability, efficiency, and applicability in real-world situations. niques applied for the software data validation phase in the automotive
industry (Fig. 1, step 1).
Fig. 1. SLR steps and phases.
3
G. Pagoto et al. Computer Standards & Interfaces 97 (2026) 104110
We created a research protocol (Fig. 1, step 2) using the Parsifal Table 3
online tool [24], which provides a template and facilitates interactive Research questions and related objectives.
collaboration approach for researchers carrying out that task [25]. The ID Research question Specific objectives
protocol defines the strategies and criteria adopted during the SLR
RQ1 What are the approaches involved To identify and classify the
execution, ensuring a standardized SLR process, minimizing biases, and in the quality assessment during software quality approaches based
improving reproducibility [23]. software data validation in the on proposed studies.
To elaborate the research questions (Fig. 1, step 3) we applied the automotive industry?
PICOC (population, intervention, comparison, outcomes, and context) RQ1.1 How are the approaches applied in To identify how the approach is
the software data validation? applied based and where the
approach as recommended by Kitchenham and Charters [22] and application focus is placed.
Wohlin et al. [26]. The PICOC terms are detailed in Table 2. RQ2 Which are the main quality To extract the main aspects or
This work is part of a master’s degree thesis for software validation assessment characteristics for characteristics taken into account
and electronic systems integration for the automotive domain. Based on software data validation in the in each study.
automotive industry?
that scenario and the PICOC, we formulated the research questions
RQ3 What are the impacts of such To identify and understand the
shown in Table 3. characteristics in the validation, influences of quality approaches on
The search strategy for this study followed the guidelines of Kitch­ releasing and using phases of the the software lifecycle.
enham et al. [27] and included the recommended databases: ACM software data validation process?
Digital Library, IEEE Digital Library, ScienceDirect, Scopus, and
Springer Link.
To identify the relevant databases and create an appropriate search Table 4
string, we conducted the Search and Selection Strategy phase (Fig. 1, Final search strings resulting from our preliminary search.
step 4). A preliminary search in Google Scholar using general terms Name Search String Results Filter
related to the topic of interest was performed to assess coverage in the
ACM Digital 26 Full year and text
research domain. We checked the results on the first four pages, looking
"automotive" AND
Library "software data" AND
for their potential relevance to our RQ’s.. (test OR validation OR
Since the automotive domain was previously defined in the PICOC verification) AND
context, the “automotive” term was included in the preliminary search ("quality assessment" OR
string. Due to a large application of terms such as “quality” and “soft­ "quality assurance") AND
(method OR framework OR
ware” in many unrelated domains (e.g. domestic equipment, audio, and model OR process)
video), we refined the search string to include “automotive industry”, IEEE Digital automotive AND quality 89 Full year and text
which showed more accurate results in some digital libraries. After this Library AND software data AND
initial exploration, we formulated a general search string -“automotive (test OR validation) AND
industry" AND software AND ("quality assurance" OR "quality assessment")
(method OR framework OR
model OR process)
AND (test OR validation) AND (method OR framework OR model OR pro­ Science@Direct "automotive industry" 69 Full text and year
cess” - using an iterative process. We conducted trial searches to identify AND "software data" AND
relevant studies and refine the search string based on their results. The quality AND (assurance
general search string was adapted to the specifics of each database, as OR assessment) AND (test
OR validation) AND
shown in Table 4. (framework OR process)
Adaptations included using database-specific operators and filtering Scopus automotive AND software 77 TITLE-ABS-KEY
options, as follows: AND (test OR validation)
AND (“quality assurance"
• Selecting the “Full text” option for most databases, except Scopus,
OR "quality assessment")
AND (method OR framework
which uses a dedicated search method for title, abstract, and OR model OR process)
keywords. Springer Link "automotive industry" 602 1. Full year,
• An initial search in Springer Link returned over 3000 papers. To AND software AND articles and
conference
refine the results, we filtered to “Articles and conference papers” ("quality assurance" OR
"quality assessment") papers
only, which reduced the results to 602 candidate studies. AND (test OR validation) 2. “Preview-only”
AND (method OR framework selected
OR model OR process)
Table 2
PICOC for research questions and rationale for each element. • In ACM and Springer Link, the original term “software data” to be too
narrow and replaced it with the broader term “software”. In the same
Population Software testing, validation, In this study, we focus on
and quality software validation, tests, and
way, we replaced “automotive industry” with the term “automotive”
quality phases within the in IEEE, ACM, and Scopus.
automotive industry. • We added the keyword “verification” in ACM, to make the query
Intervention Methodologies, processes, Refers to the environments and more specific and retrieve additional studies.
frameworks, models, methods, methods applied, such as
and techniques methodologies, processes, We did not limit the studies by publication period in the search
frameworks, models, methods, phase. We found out, in our preliminary searches, that the quality
and techniques
Comparison Not applied Not in the scope of this SLR.
assessment for safety-related software has been reported from 1990
Outcome Quality assessment Expected results include quality onwards. The first study concerned with safety in control functions is
characteristics, performance assessment characteristics, dated 1993, and out of 28 studies found before 1993, none focused on
indicators, and validation performance indicators, and software quality assessments. Thus, we applied the exclusion criteria “E2
impacts impacts on validation.
- Older than 1990s” during later screening. A complete list of our in­
Context Automotive industry The application domain is
restricted to the automotive clusion and exclusion criteria is provided in Table 5.
industry.
4
G. Pagoto et al. Computer Standards & Interfaces 97 (2026) 104110
Table 5 Table 6
Inclusion and Exclusion Criteria. Selected studies by title and abstract.
ID Inclusion Criteria Source Candidate papers Included by Included in full
(database search screening title and text screening
I1 Primary studies;
results) abstract
I2 Papers that consider software data validation/test phases in the automotive
industry AND: ACM Digital 26 2 (7.7 %) 2 (7.7 %)
I2.1 Discuss methodologies, processes, frameworks, methods, tools, or models Library
of quality assessment/assurance for software data; OR IEEE Digital 89 52 (58.4 %) 23 (25.8 %)
I2.2 Papers that address the impacts of quality assessment on software data Library
ID Exclusion Criteria: Science@Direct 69 5 (7.2 %) 1 (1.4 %)
E1 Duplicated studies; Scopus 77 35 (45.5 %) 17 (22 %)
E2 Older than 1990′s; Springer Link 602 63 (10.5 %) 30 (5 %)
E3 Papers that consider software development but not in automotive industry; Total 863 157 (18.2 %) 73 (8.4 %)
E4 Papers not related to software validation/test phases;
E6 Papers that are not articles from journals, conferences or magazines
E7 Papers that are not related to vehicle-embedded software • RLC circuits testing, verification, and validation (E7). Papers strictly
E8 Conference Proceedings
focused on hardware and without qualitative contributions for the
E9 Papers not written in the English language
E10 Not accessible in specific databases
scope of SLR.
• Hardware requirement approaches (E7 and E4). The requirements
for hardware, presented in some studies, did not have a relevant
3.2. Conducting impact on software validation approaches. We opted for selecting
software-specific studies when they cite the requirements for soft­
To assess the understanding of inclusion and exclusion criteria ware in vehicle-embedded systems and present impacts on software
among the researchers, we performed the Pilot Selection (Fig. 1, step 5) - quality assessments (I2.2).
two rounds of individual selection and agreement check with a sample of • Description of software tooling quality evaluation (E7). The Software
20 and 24 random candidate papers, respectively. We held meetings Assessment (SA) approaches for test tooling are out of this SLR scope.
with all researchers aiming to identify the divergences and harmonise • Data science solutions for IT software (E7 and E4). The data science
the criteria understanding. We performed two additional discussion application is not in scope unless the focus is placed on automotive-
rounds to clarify divergences and doubts regarding study selection. In embedded software evaluations during validation phases.
the first one, we found cases within the automotive domain involving • Software quality processes applied for production plants (E7 and E4).
software validation, but not specific in-vehicle embedded software, Several production software solutions are important for the software
which is the target of our research. For example, some papers addressed quality lifecycle, and they are out of SLR scope because of their
the QA of external sound equipment installed in the vehicle, which is out contributions after software validation phases and release for series
of scope. To solve this issue, we added the exclusion criteria “E7- Papers production.
that are not related to vehicle embedded software” and “E8-Conference
Proceedings”. In the second round, we excluded extra papers based on As proposed by Kitchenham and Charters [22], we performed the
the criteria “E9- Papers not written in English language” due to the low Quality Assessment (Fig. 1, Step 7), aiming for an improved interpre­
relevance of only two non-English papers identified. tation of selected studies. The quality assessment questions presented in
To assess the agreement level among the researchers and ensure a Table 7 are based on SLR objectives and aimed to identify relevance for
rigorous selection, we computed the inter-rater agreement proposed by selected research questions (RQ’s). The rationale behind those QA
Landis and Koch [28]. The results were: questions is described in Appendix A. The possible answers are 1 point
(or “Yes”), 0.5 points (or “Partially”), and zero points (or “No”).
• Pilot selection 1 (n = 20): k-factor = 0.342 (Fair). A quality assessment of the selected studies was carried out after the
• Pilot selection 2 (n = 24): k-factor = 0.750 (Substantial). full-text screening. To reach proper representativeness and quality re­
sults, we adopted a cutoff score greater or equal to 6.0. Thirteen out of
After the first round, referred to as “pilot selection 1″, we reviewed 73 candidate papers below this threshold were further excluded, many
and discussed the disagreements. Out of 20 randomly sampled studies, of which do not fulfil our quality assessment questions Q6, Q7, and Q8.
five disagreements occurred due to unclear inclusion/exclusion criteria. Fig. 2 shows the distribution of studies by achieved score.
Specifically, the validation or test phases for automotive embedded
software were not always clearly distinguished, and other out-of-scope
cases, e.g., software applied in a vehicle assembly line or software for Table 7
supporting vehicle development processes. After the researcher’s dis­ Quality Assessment Questions.
cussion and refinement of exclusion criteria “E4” and “E7”, we per­ ID Question
formed the “pilot selection 2”, resulting in a substantial improvement in Q1 Is the study related to the proposed SLR objectives?
agreement. Q2 Does the study bring relevant information for answering research questions?
We then started the Studies Selection phase (Fig. 1, step 6). We Q3 Is the quality application field clearly described and applicable to vehicle
embedded software? (RQ1)
screened the candidate papers by reading the title and abstract, applying
Q4 Are the methodology, processes, models or framework clearly presented?
the inclusion and exclusion criteria listed in Table 5. In case of uncer­ (RQ1.1)
tainty, a full text screening was conducted. Out of 863 candidate papers, Q5 Is there any technique comparison included? If yes, is that clearly presented
we included 157 (18,2 %) based on title and abstract. Table 6 describes and described?
the distribution of included and excluded studies by electronic database. Q6 Does the study present any real use-case investigation or is applicable for?
(RQ2)
We further excluded 84 papers in full text screening that met the
Q7 Does the study bring consistent results regarding the presented technique(s)
exclusion criteria E7 - “Papers that are not related to vehicle embedded approach? (RQ3)
software” and/or E4 - “Papers not related to software validation/test Q8 Does the study present success, unsuccess examples or issues during the
phases”. Table 6 also details the distribution of final selected studies software validation phase?
Q9 Is the related study applicable to the commercial vehicles segment?
after the full text screening in the second selection round. We presented
Q10 Does the study bring any coverage of quality issues related to software
below some examples to illustrate the researchers’ rationale: validation lacks?
5
G. Pagoto et al. Computer Standards & Interfaces 97 (2026) 104110
Fig. 2. Distribution of papers based on the achieved score.
Fig. 3 shows the quality of included and excluded papers by publi­ • Proposed solutions (P1): The proposed solutions for software data
cation year. Papers published until 2005 accounted for <7 % (3 papers) validation were categorised in the following groups: case study,
of our dataset, whereas a significant concentration of studies emerged in framework, methodology/method, model, process, technique, and
the last decade. We observed an increase in high-quality studies in tool.
recent years, possibly linked to stricter regulatory requirements and/or • Software Assessment type (P2): We analysed the focus of studies in
the growing maturity of research fields such as safety-critical and terms of quality, safety, security, performance, or any other moti­
cybersecurity domains. The growth trend identified cannot be fully vation for a proposal development solution.
confirmed for the last period (2021–2024), as the period is not yet • Software Assessment phase (P3): The objective here is to discern
complete, and the collection for the studies concluded in the first quarter the software lifecycle phase, in accordance with Sommerville [29],
of 2024. within which the software assessment is applied: concept, design
The application of the protocol is summarized in the flowchart in (including implementation), validation, development phases (except
Fig. 4. The specific data for each database is described in Appendix A. maintenance), or complete lifecycle coverage (including
We then carried out the Data Extraction phase (Fig. 1, step 8), maintenance).
considering a set of data characteristics (presented in Table 8) linked to • Involved software data source (P4): To understand the possible
the research questions, here called properties (from P1 to P8). An paths and origins of software data, we extracted existent sources
additional property, P9 was added later due to the relevance of new from which the software data is validated. Sources found dataset,
technology trends for software data assessment. dynamic tests, static tests, modelling/simulation, source code, and
specification.
Fig. 3. Quality of studies by publication year.
6
G. Pagoto et al. Computer Standards & Interfaces 97 (2026) 104110
• Application of Artificial Intelligence (AI), Neural Network (NN)
or Machine Learning (ML) techniques (P9): We intend to check
how much the application of such techniques is cited to improve the
software quality and provide support to the developers to mitigate
the failures in the field.
The categorized groups and their definitions were as stated by the
authors of the primary studies. Our classification was based on findings,
and we did not intend to re-classify them into different definition
groups, rather, we retained those stated in the primary studies. Appen­
dix A lists the extracted terms for property groups used in the data
extraction phase.
For the Data Synthesis (Fig. 1, step 9), we applied qualitative analysis
and inductive coding to analyse the findings of the studies [30]. The
inductive coding process allows the creation of codes directly from the
collected data, instead of relying on a predetermined coding framework
[31]. Each code represents similar textual contents in the studies and
groups them according to the assigned category. When necessary, a
second round of coding was conducted to merge similar codes into a
common final categorization. Fig. 5 shows an example of the coding
process.
3.3. Reporting
The Reporting phase (Fig. 1, steps 10 to 13) are presented in sections
4 (Results and Analysis), 5 (Discussion, Challenges, and Trends), and 6
(Conclusion).
Fig. 4. Summarized protocol application.
3.4. Threats to validity and limitations
Table 8 The term “quality” is largely applied to several areas of knowledge.
Extraction list for properties. In the software data validation and quality assessment process domains,
ID Property Research
we noted a large variation in how this term is used. The complexity and
Question potential restrictions of the defined string may also have impacted the
identification of relevant papers. We attempted to mitigate these risks by
P1 Proposed approaches/solutions RQ1
P2 Software Assessment (SA) type RQ1 increasing the rigour during the studies selection phase (step 6), per­
P3 Software Assessment (SA) phase RQ1.1 forming meetings and rounds of discussion among the researchers to
P4 Involved software data source RQ2 reach a more assertive result. In the same way, the decision to state a
P5 Other industry areas, segments, or product types than RQ1.1 publication date limitation after the title and abstract readings
automotive also cited
P6 Main software characteristics considered RQ2
contributed to accessing all available papers and checking their real
P7 Main assessment characteristics source RQ2 relevance before any exclusion step.
P8 Main impacts of quality assessment RQ3 During the first selection phase, we noted several cases of quality
P9 Application of Artificial Intelligence (AI), Neural New technology assessment analysis focused on testing tools, which is out of our research
Network (NN) or Machine Learning (ML) techniques trends
scope. To exclude these papers, we added the exclusion-related criteria
for non-embedded software-related papers (E7 in Table 5). This solution
• Cited industry areas, segments, or product types others than mitigates but does not eliminate completely the chance of finding
automotive (P5): The Software Assessment (SA) in the automotive studies mentioning testing tools in the context of embedded software
domain can bring relevant approaches valid for many other domains. quality assessment. In such cases, we opted to keep the papers that
This fact moved us to check which are the most cited in the selected contributed to answering the research questions.
studies. The possible categories are aerospace, avionics, chemical, In Section 4 - Results and Analysis, we used classification terms as
automation, rail systems, nuclear, and software. stated by the primary studies. That means our classification was based
• Main software characteristics considered (P6): The target is to on direct findings, and we did not reclassify the studies based on our
identify the software data characteristics described in the studies judgment. We recognise that overlaps or similarities among the different
during the SA phase. We define such characteristic groups by terms may introduce biases originating in the primary studies, which
applying the coding after the data extraction, without predefined could reflect in our results.
categories. We conduct this analysis to support us in answering RQ2
properly. 4. Results and analysis
• Main Quality Assessment (QA) characteristics source (P7):
Similar to P6, but with the focus on identifying the origins and ref­ A total of 60 studies were included and considered for data
erences of the studies to perform a software data assessment extraction.
approach.
• Main impacts of Quality Assessment (QA) (P8): The assessment 4.1. Software quality approaches based on proposed studies (RQ1)
performed with a target to validate a software, in general, results in
positive impacts such as cost, safety, security, error mitigation, per­ The main software quality approaches stated in the included studies
formance, and user experience. are Techniques (25 %), Methods (21.7 %), Frameworks (18.3 %),
Models (16.7 %), Processes (6.7 %), and Case studies (5.0 %). Other
7
G. Pagoto et al. Computer Standards & Interfaces 97 (2026) 104110
Fig. 5. Coding process examples for P2.
categories (Methodology, Standard, Systematic Mapping Study, and methodologies of sensor networks and stream processing to optimise the
Topology) were mentioned just once (1.7 % each). amount of data to create a method for selectively preprocessing and
The predominance of Techniques and Methods (together over 45 %) recording measured data, which is the main intended target of their
suggests a practitioner-driven focus on implementation-ready tools. study. Along the same line, Touw [12] performed a case study merging
Frameworks and Models are academically grounded, but they might three assessment models (CMMi, TMMi and A-SPICE) to establish a new
demand higher integration or customization effort. This indicates a model taking account of the quality aspects of complex systems.
prioritization of practical solutions in safety-critical automotive envi­
ronments. The complete number of studies per software quality 4.1.1. How the software quality approach is applied and where the
approach is illustrated in Fig. 6. Models. Models, as described by Tak­ application focus is placed (RQ1.1)
rouni et al. [32], are based on other already existing models to extend The purpose of this research question is to understand how the
the applicability and cover the complexity of modern vehicles. This is software quality approach was applied in terms of the SA type domains.
possible through the addition of modules, focused on desired coverage We found six different target groups, as illustrated in Fig. 7, many of
areas, such as, reliability, safety, cyber security, cost optimization and them focused on a single quality characteristic. Several studies consid­
others. ered a combination of quality characteristics coming from multiple
Techniques. Mateen et al. [33] propose a technique for an analysis domains that bring a better compromise among performance, time, cost,
work to provide a better way to evaluate the software quality level, use of technology limits, improved user experience, and others. For
having the comparison between manual testing vs automated testing example, Sadio et al. [39] defined quality based on proper detection of
aiming the minimum error rate within the time and lower target costs. Wi-Fi power thresholds, which improve customer experience and
Techniques based on software prediction are also cited in [34–37]. satisfaction.
Ranasinghe et al. [35] emphasise that the quality of intelligent auton­ Fig. 8 points out Safety and Performance as the most relevant char­
omous decision-making can be reached by applying techniques of the acteristic for software quality evaluation, independent of the adopted
fusion of sensor data and historical health-state information. approach. In the automotive domain, those attributes are often inter­
A mix of software quality approaches could be also found in [38], related. For example, the capacity of a front radar for autonomous sys­
comprising failure detection via two diagnostic protocols, vehicle data tems to maintain the coverage angle against powder, snow, exhaust
measurement, fault monitoring, and unexpected behaviour analysis. The gases, water, and fog is directly related to the safety-critical system
authors proposed an application method based on existing operation. At the same time, a safety assessment for safety-critical
Fig. 6. Distribution of studies by software quality approach.
8
G. Pagoto et al. Computer Standards & Interfaces 97 (2026) 104110
Fig. 7. Distribution of studies by SA type.
Fig. 8. Distribution of Assessment Focus (P2) by Software Quality Approaches (P1).
systems requires that the correct sensor data, processed by the software, did not describe clearly the source of their SA data. Some reasons for that
is correct and available. are their focus on (1) the effects of the adopted approach or presented
Taking account of the classification by SA phase (P3), we identified solution [40], (2) the comparison of approaches [33,36] or other similar
the software lifecycle phases in which software QA is placed (shown in targets. Other studies [41–45] reported more than one source of data.
Fig. 9). Not surprisingly, most of the studies are focused on the Valida­ The most relevant software quality approaches (techniques,
tion phase (75 %) but others also mentioned Concept (11.7 %), Full methods, frameworks, and models) represent 81.7 % of the total selected
lifecycle (5.0 %), Software development (3.3 %), among others. The studies. These approaches can often be implemented in combination, as
predominance of Validation phase suggests that QA is often placed be­ proposed by [11,32,38]. The main reported motivation for adopting
tween Design and Deployment phases. This may reflect industry prac­ combined approaches is the need to fit them to a specific new demand
tices where late-stage validation is still a primary gatekeeper for such as increasing systems complexity [13,44,46,47] or cost [36,48,49].
compliance and cost, despite calls for shift-left approaches. Oka et al. [50] describe another solution that integrates tools and test
We also examined data sources for software QA (P4) to identify the cases. The proposed process uses information from Application Lifecycle
origins of validation data, as illustrated in Table 9. Ten studies (16.7 %) Management (ALM) tools to configure the testing tools and, finally,
9
G. Pagoto et al. Computer Standards & Interfaces 97 (2026) 104110
Fig. 9. Distribution of studies by software QA application phase.
assessment in terms of product type segment, as illustrated in Fig. 10. In
Table 9
second place, autonomous systems (16.7 %) are increasing in relevance
Software data sources.
due to higher efforts concentrated over the last years, as pointed out by
Software data source Occurrences [7,13,14,35,37,52–56]. The electronic control system (13.3 %), and
Specification 21 (35 %) Powertrain (6.7 %) completed the most relevant product type segments.
Dynamic test 13 (21.7 %) The dominance of embedded systems suggests that a deterministic
Not clearly described 10 (16.7 %) behaviour could be more suitable for QA. The growth of sophisticated
Dynamic and Static tests 3 (5 %)
Source code 3 (5 %)
software products to attend very complex computer systems requires
Data validation set 2 (3.3 %) that QA starts as early as possible, thus reducing the correction costs [17,
Modelling 2 (3.3 %) 36,42,57,58].
Static test 2 (3.3 %) In Fig. 11, the Validation and Concept phases lead to the software
Dynamic test and Specification 1 (1.7 %)
Development phases, but their relevance is not the same from a product
Historic fault data 1 (1.7 %)
Simulation 1 (1.7 %) type perspective. The presence of the Concept phase for the Embedded
Static test and Specification 1 (1.7 %) Software, Electronic Control, and Autonomous Systems indicates that
the assessment focus is also centered on base software or code, instead of
the Powertrain. Assessment of Powertrain primarily involves evalua­
collect the test results back to the ALM tools. tions of acceptance characteristics, like gear shifting software charac­
Similarities with other sectors (P5), such as aerospace, avionics, rail teristics (comfort, shifting speed, gear scratches), more traditional
systems, chemical, and automation were cited by four (6.6 %) studies mechanics evaluation approaches, and driver’s feeling. Assessment of
[11,35,40,51]. The reasons for this are the increasing relevance of safety Electronic Control products focuses on microcontroller management,
in automotive software applications (e.g. autonomous driving), cyber­ memory, drivers, standards, and proprietary communication. In the
security requirements, high-voltage applications for battery-electric Autonomous Systems, the balance between Validation and Concept is
vehicles, and new standards linked to connectivity and over-the-air expected to increase in the next years due to the importance of safety-
applications. critical systems issues being addressed early. Rework and additional
Furthermore, embedded software (48.3 %) leads the software data costs, due to late identification of defective software, can be a no-go for
Fig. 10. Product type segments reported by the studies.
10
G. Pagoto et al. Computer Standards & Interfaces 97 (2026) 104110
Fig. 11. Concept and Validation Phases by Product Type.
the start of production releases. autonomous driving, e-mobility applications, and telematics. Further­
more, other cited characteristics (code quality, misfunction, perfor­
4.2. Main aspects or characteristics considered in each study (RQ2) mance, communication quality, data accuracy, data interpretation, and
software health) will also be affected by those standards and regulations.
The main assessment characteristics considered in QA for software In terms of data sources, we identified the following origins: speci­
data validation are the scope of this research. Furthermore, we also fication with 22 occurrences (36.7 %); requirements with 14 (23.3 %);
intended to identify the origins of such characteristics and understand measured data with seven (11.7 %); fault data with three (5.0 %);
their relationship (P4, P6, and P7). simulated with two (3.3 %); CAN databases, emulated data, source code
Out of 60 included studies, 17 (28.3 %) mentioned code quality. The and stored project data with one (1.7 %) occurrence each one. Eight
main cited characteristics are Reliability, Maturity, Maintainability, studies (13.3 %) did not report a specific data source or did not simply
Fault Tolerance, Availability, Safety, Cybersecurity, and Security. emphasise that. This could be because some internal documents are
Despite the automotive domain’s particularities, they are mainly placed confidential or protected by non-disclosure agreements.
on the requirements specification, not in the software. This means that a Lastly, but not less important, we tracked software data sources.
QA for software data validation can be established by generic software Fig. 12 shows the distribution of the occurrences for each software data
quality standards such as ISO/IEC25010, where Functional Suitability source. The majority of studies concentrates in specification (21 occur­
can also cover the Functional Safety (FuSa) automotive-specific re­ rences, 35 %) and dynamic and/or static tests (20 occurrences, 33.3 %).
quirements [46]. Ten (16.7 %) of selected studies did not clearly mention the applied
From the characteristics illustrated in Fig. 7, we can highlight that source for QA. The causes for this may be the same as described above
Safety and Cybersecurity have been increasingly cited in recent years. A for reported data sources.
potential explanation for this is the spread of regional regulations
implementation, such as UN-ECE R155 for Cybersecurity and UN-ECE 4.3. The impacts of QA approaches on the software lifecycle (RQ3)
R156 for software update management, and new standards, such as
ISO26262 – Functional Safety, and ISO21434 – Cybersecurity, to It may be obvious that the main impact of a well-performed software
establish a baseline for new complex upcoming technologies such as QA process is improved quality itself. As expected, the word “quality”
Fig. 12. Reported sources for software data.
11
G. Pagoto et al. Computer Standards & Interfaces 97 (2026) 104110
was the most cited impact, it was reported by 22 studies (36.7 %). phase, AI, ML, and NN can be widely applied. Some examples of their
We aimed in RQ3 to identify other potential impacts and understand use are as follows:
how they are perceived. Thus, we analysed additional cited terms Artificial intelligence (AI). In addition to conventional vehicle
beyond “quality”. The complete overview of the reported impact dis­ tests, AI can be used for automated test generation and creation and
tribution is presented in Fig. 13. failure-revealing scenarios for safety-systems evaluation, as reported by
The reported techniques, frameworks, and methods are mostly [37].
applied to safety-critical systems. Autonomous systems are predominant Machine learning (ML). System-healthy assessment can be opti­
(e.g., brake assist, adaptive cruise controller) as well as safety-related mised by ML models, as pointed out by Ranasinghe et al. [35]. For
embedded software products (ISO26262 compliance). These reported example, ML has been used for state of charge (SoC) battery estimation
characteristics reinforce the concern of automotive software developers in blind modelling tools [41]. Another example is the use of data-driven
in recent years. The complete distribution by approaches is presented in ML for dual-clutch transmission software, where input data - including
Fig. 14. actuator constraints and performance/comfort characteristics – are used
Table 10 presents the comparison between the SA type (P2) and the to evaluate optimal gear-shifting settings [34].
main perceived impacts on Quality Assessment (P8). As illustrated, there Neural network (NN). A variant called Recurrent Neural Network
is a clear link between the main perceived impacts (P8) and the software (RNN), described by Kollmeyer et al. [41], is a type of supervised ML
quality assessment type (P2). Ranking first in both is quality, followed method that uses short-term memory to process battery state of charge
by safety, and performance in third. The key messages here: 1) Efforts (SoC) quality metrics. It can be used for time-dependent applications
can be applied directly to collect the desired outcomes (quality, safety, with sequential datasets, where the goal is to apply the best-trained
performance, and cybersecurity); 2) Side effects are not restricted to the network in the next processing step. Another study [3] trained NNs
directly aimed SA type, the benefits extend further; and 3) QA can also with quality parameter metrics to evaluate the software quality and
have positive impacts on other aspects, such as user experience, effi­ produce performance ratings for Powertrain applications.
ciency, and cost. The distribution of quality assessment characteristics by these tech­
The growing emphasis on cybersecurity aligns with regulatory shifts niques is as follows: Safety level with three occurrences (33 %), code
such as UNECE-R155 and UNECE-R156. This points out concerns with quality and performance with two each (22 %), data accuracy and state
public trust and liability management. Despite their lower impact of health one each (11,1 %).
ranking on QA (7th position), cybersecurity topics are now more cited These techniques are suitable for high-complexity systems during
than UX as SA types, revealing an increasing priority gap between se­ Verification & Validation (V&V) activities. However, their application
curity and user-centred validation. present challenges for software developers, such as higher costs due to
In terms of cost and efficiency, even though they are not directly additional required efforts, the need for advanced infrastructure and
cited in the SA phase as primary targets, they remain an integral part of expertise. Their focus on autonomous and other safety-critical systems
the entire software lifecycle and listed by studies as perceived impacts in makes sense, as these areas demand superior product quality and reli­
the 5th and 6th positions, respectively. ability that traditional validation processes cannot achieve [36]. Addi­
tionally, their scalability can mitigate undesired events during tests (e.
4.4. Application of new techniques and their benefits for software quality g., accidents) and minimize the need for physical tests, reserving pro­
assessment totype vehicle trials in controlled or public road conditions only at
higher software maturity levels.
Out of 60 included studies, nine (15 %) mentioned at least one of the In terms of applicability, these techniques can support several soft­
following techniques: artificial intelligence (AI), machine learning (ML), ware and data quality assessment tasks. For software code, they aid in
and neural networks (NN). Those upcoming trends are also relevant to specification from requirements, code creation through reuse, identifi­
mention. cation of code deviations from fault storage data, test monitoring, and
In terms of product type segment, four out of nine (44 %) focused on result analysis. For software functions, especially the safety-related
autonomous systems. These studies emphasized the validation ones, benefits include driver behaviour recognition (e.g., detecting fa­
complexity and the impact on fulfilling functional safety requirements tigue, misuse of cell phones while driving, etc.). They can also extend
[7,14,35,37]. Two studies (22 %) addressed Powertrain systems, high­ into post-production, enabling quality experts to monitor the software
lighting relevant challenges in the integration among electronic, elec­ quality performance in end-user applications and apply proper correc­
trical, and mechanical components (e.g., automated manual tions when needed.
transmissions). Electric vehicles were cited in one study (11 %), as was However, there are constraints. AI- and ML- techniques face chal­
general embedded software (11 %). lenges in meeting safety-critical requirements in real-world conditions
Regardless of the intended characteristic being evaluated in an SA [7]. This limitation prevents the exclusive application of those tech­
niques without additional efforts and funding for complementary vali­
dation in the “real-life” scenarios. Furthermore, variations in product
characteristics can affect system responses, requiring a worst-case
evaluation to identify critical variants for testing and ensure a proper
functional performance while mitigating safety and security risks.
5. Discussion, challenges & trends
Many studies focus on quality-specific metrics but fail to compre­
hensively address all the characteristics of inherent and system-
dependent software quality assessment as defined by ISO/IEC25012.
While data accuracy, data interpretation (often referred to as “under­
standability”), and efficiency are frequently cited, terms like “code
quality” can also align with this international standard as part of a more
comprehensive software quality assessment approach. This is further
supported by studies that identify safety (ISO 26,262) and cybersecurity
Fig. 13. Software QA reported impacts. (ISO21434) as key assessment characteristics, with “compliance” being
12
G. Pagoto et al. Computer Standards & Interfaces 97 (2026) 104110
Fig. 14. Perceived QA Impacts (P8) by Software Quality Approaches (P1).
standards are further regulated by regional implementations, such as
Table 10
UN-ECE R155 for Cybersecurity and UN-ECE R156 for software update
Comparison between SA type (P2) and Main perceived impacts of QA (P8).
management. They apply particularly, though not exclusively, to safety-
P2 - SA Type P8 - Main Impacts of QA critical features such as autonomous systems, electric propulsion, and
Quality 1◦ 1◦ Advanced Driver Assistance Systems (ADAS). Therefore, to ensure
Safety 2◦ 2◦ proper code quality, all the aforementioned aspects must be considered
Performance 3◦ 3◦ from the software concept phase. If these characteristics are neglected,
User Experience (UX) - 4◦
Efficiency - 5◦
they can negatively impact not only software quality but also safety,
Cost - 6◦ security, and compliance with regional regulations.
Cybersecurity 4◦ 7◦ Impacts on the software lifecycle. Quality was identified as the
Traceability 5◦ - most significant impact throughout the software lifecycle. Ensuring
Reliability 6◦ -
software quality over its lifetime involves focusing on traceability,
applying quality tools (such as FMEA and FTA), and implementing
a particularly notable focus. Another group of studies emphasises safety approaches. The second most significant impact is Safety, which
portability, reliability, security, user experience (often referred to as arises from software quality assessments that prioritise safety and per­
“usability”), and performance, all based on the principles of the product formance. This is reinforced by functional safety requirements outlined
quality model established by ISO/IEC25010. This model serves as the in ISO26262. Additionally, User Experience (UX) was reported as an
foundation for software requirements specification and quality important factor, encompassing all user perceptions of the final product,
evaluation. which can greatly influence customer satisfaction. Attention to UX has
Best practices in software development processes significantly increased in recent years with the incorporation of interactive solutions,
enhance the efficiency of quality assessment, especially given that such as digital and touch screens, connectivity features, and voice
approximately 85 % of vehicle functionalities are software-based today. interaction.
We examined quality characteristics and the impacts of these The findings of RQ1 brought an overview of applied approaches, the
assessment approaches on the software lifecycle, as detailed below: most relevant targets, and motivations for an automotive software
How each approach is applied and where the application focus quality evaluation. The contributions of RQ1.1 provided the automotive
is placed. The distribution of software quality assessment approaches software segment groups and identified the phases of software devel­
was concentrated in Techniques, Methods, Frameworks, and Models. opment where the approaches are concentrated. The RQ2 comprises the
Notably, these categories are not dependent on the specific application characteristics considered, the software source data (acquisition type),
focus. The three most relevant aspects identified are: Quality, Safety, and the provided sources for those characteristics (requirements, stan­
and Performance. Regarding the application phase, Validation was cited dards, and/or specifications). Finally, the RQ3 improves the under­
in 75 % of the 60 studies included in this review. Other phases, such as standing of perceived impacts and results of a well-implemented
Conceptualization, Development, and Field Application, are expected to automotive software quality assessment.
increase in relevance, particularly due to the complexity of cyberse­
curity and safety-critical systems that necessitate monitoring during the 5.1. Challenges & trends
process to enhance failure prediction during vehicle operation.
Quality characteristics. Code quality was the most frequently cited The studies also revealed several challenges and issues, such as:
characteristic, as the structure of the software code is critical to various
aspects, including complexity, efficiency, execution time, and reliability. • Ensuring safety-critical requirements: Addressing these chal­
Additionally, code quality is intrinsically linked to other characteristics lenges in real-world conditions, particularly with the introduction of
such as malfunction, performance, communication quality, data accu­ new AI and ML techniques, requires identifying and clarifying safety
racy, data interpretation, and software health. and fail-operational requirements as the foundation for an effective
The other two significant topics identified were safety level and testing strategy [7]. The requirements specification should ensure all
cybersecurity level, both of which are guided by standards like intended application environments and cover the possible software
ISO26262 (Functional Safety) and ISO21434 (Cybersecurity). These variances during the entire lifecycle.
13
G. Pagoto et al. Computer Standards & Interfaces 97 (2026) 104110
• Adapting to rapidly changing software requirements and of tools for software and systems management, such as the Engi­
environments: neering Lifecycle Management (ELM) that provides support for re­
• Frequent changes hinder alignment with standards and processes, quirements, tests, and design specifications. This integrated
leading to increased costs and efforts in correcting software defects. documentation environment enables the software development
To facilitate early defect detection, a risk-based approach that teams to deal with complexity, software compliance, and auditable
identifies high-potential failures is essential. This should be sup­ documentation.
ported by exploratory testing based on ISO 29,119 [59]. The soft­
ware documentation, including requirements, code specification, While these findings are centred on the automotive domain, they can
tests, and related reports, must be maintained. Lifecycle modes, like also benefit other industries, such as aviation, electronic control sys­
agile, support the software development teams in this challenging tems, automation, railway transport, and autonomous systems, by
task. This does not, by itself, prevent software issues but contributes enhancing their software quality processes through similar or adapted
to a structured basis for improved software traceability throughout approaches.
the whole development phase.
• Bridging the gap between simulated and physical systems: 6. Conclusion
Simulations are often constrained by what is known as the “reality
gap.” This issue can be mitigated by incorporating real measure­ Higher software quality positively influences customer acceptance of
ments into the simulation and validating the relevant components of a product. Over the past 20 years, the term ’software quality assessment’
the physics engine using modelling software [52]. Additionally, the has evolved to include new assessment criteria to support the automo­
use of digital twins - virtual representations of physical systems tive software developers in managing growing systems’ complexity. The
updated with real-time data - can further narrow this gap. It allows primary reason for this evolution in the automotive industry is the rapid
continuous validation of existing physical tests using real-world expansion of complex technologies, including Advanced Driver Assis­
measurements in prototype vehicles or customer fleets (under an tance Systems (ADAS), connectivity systems (such as over-the-air fea­
official customer’s agreement) via upcoming over-the-air technolo­ tures), electric propulsion vehicles (e-mobility), battery electric vehicles
gies for data acquisition. The existing infrastructure may be insuffi­ (BEV), and hybrid electric vehicles (HEV). To effectively address the
cient to fulfil the requirements for the validation task, and challenges posed by this complexity, the quality assessment must align
complementary funding shall be considered only for “in-vehicle” with new requirements and standards. Additionally, a more compre­
equipment, not for the acquisition of infrastructure from scratch. hensive approach can enhance understanding and better equip software
• Improving quality assessment testing procedures and metrics: developers and testers to tackle the challenges.
The absence of clear software quality assessment and testing metrics This SLR identified 60 primary studies on automotive software
in the automotive industry results in vulnerability issues. To address quality assessment published over the past 30 years. It synthesized the
this, a weight-based approach targeting the most vulnerable software findings about four main types of approaches (techniques, methods,
components is proposed [53]. The application of robust approaches frameworks, and models) and their focus across the software lifecycle,
based on quality, safety, and security standards, internal docu­ with Validation being the most commonly addressed phase (75 %). The
mented lessons learned, and state-of-the-art practices certainly con­ review also mapped QA characteristics, with the most cited being code
tributes to improved software quality. Nevertheless, the accumulated quality (28.3 %) and safety (20 %). Safety and cybersecurity (3.33 %)
knowledge, expertise, and adequate resources for continuous pro­ figure in alignment with standards such as ISO 26,262 and ISO 21,434. It
cesses optimization are critical success factors for the long-term for also revealed that AI and ML techniques are emerging in the field,
software quality. though currently applied in only 15 % of the studies, mostly in auton­
• Reducing cost and effort in safety-critical embedded software omous and powertrain systems. Finally, our SLR offers actionable
quality assessment: A set of software defect prediction approaches guidelines (Section 5) for practitioners and researchers to improve
to mitigate these impacts is presented in [36]. Reducing the quality future software quality processes by applying standards like ISO/IEC
assessment does not mean, in principle, eliminate spending, but 25,010 and 25,012, and regional regulations such as UN-ECE R155 and
allocating it properly. As aforementioned, the implementation of a R156.
robust software validation process reduces the rework costs. In We explored the software quality approaches used in the automotive
addition to that, flexibility for corrections, modularity for the addi­ industry, clarified their applications, and identified their specific areas
tion of new software functions blocks, and applying the defect of focus. Our results indicate that new software developers should adopt
detection techniques as early as possible, can contribute to miti­ a more comprehensive quality approach throughout the entire software
gating the final software validation costs. development process, beginning with the concept phase, to enable
• Addressing problems in safety-critical machine learning sys­ earlier detection of non-quality issues, reduce correction costs, and
tems: Kuwajima et al. [14] emphasised the need for improved avoid showstoppers on vehicle assembly lines. The quality characteris­
data-driven requirements specification and design. Both test data tics and impacts outlined in this SLR ensured through the use of quality
and training data should accurately reflect real application scenarios. standards (ISO25000 series), proper specifications and requirements,
Additionally, they propose future research to address gaps in current encompassing key pillars such as safety, cybersecurity, user experience,
quality standards, such as ISO/IEC 25,000. As pointed out for the gap and performance. Additionally, regional regulations and their impacts
between simulated and physical systems, the use of real-world data on technical specifications must be considered during the conceptuali­
can stimulate and teaching machine learning systems. The combi­ zation phase. Quality tools remain essential for the entire software
nation of measured data and simulated environments can be pro­ development lifecycle. Furthermore, it is crucial to involve customers as
vided, for example, a more comprehensive scan of radars and much as possible in the conceptualization phase and to gather their
cameras from an active brake assist system. The integration of such feedback during the initial pre-series (pilot production) evaluations to
benefits can also provide a reduced development cost by shared enhance user experience (e.g., gearshift strategy for automated trans­
resources. mission software). The relevance of AI, and ML in managing complex
• Overcoming poor documentation and scalability challenges: technologies is evidenced, but their integration depends on traditional
Software testing strategy decisions are often based on experience, real-world validation, which introduces safety risks. The availability of
and unclear textual specifications can negatively impact software expert workforce, infrastructure costs, and day-by-day improvements
quality. Therefore, improved documentation practices are essential will likely define which organizations will stay and which ones will
for enhancing software quality management. An example is the use perish in this domain.
14
G. Pagoto et al. Computer Standards & Interfaces 97 (2026) 104110
The SLR outcomes can be used as a guideline for software developers through a comparative analysis of emerging studies and traditional
to implement and/or improve the automotive SA approaches by using approaches.
documentation-supported tools (e.g.: ELM), focusing on the most rele­
vant characteristics highlighted in state-of-the-art practices, and aligned CRediT authorship contribution statement
with recommended quality, safety, and security standards.
We encourage the research directions focused on how the more Gilmar Pagoto: Writing – review & editing, Writing – original draft,
comprehensive real-world data analysis can contribute to mitigate the Methodology, Investigation, Conceptualization. Luiz Eduardo Galvão
existing test modelling and simulation gaps, especially for safety-critical Martins: Writing – review & editing, Validation, Supervision, Method­
automotive software (e.g., autonomous systems). Second specific ology, Formal analysis, Conceptualization. Jefferson Seide Molléri:
research is a deeper investigation into the potential contributions of the Writing – review & editing, Validation, Methodology, Formal analysis,
over-the-air automotive data acquisition techniques for AI-based sys­ Conceptualization.
tems training activities, focused on software quality assessments.
Furthermore, the research in the aforementioned techniques – AI and
ML– to gain deeper understanding of future trends in supporting safety- Declaration of competing interest
critical quality assessments. Comparison with existing systematic re­
views can support this goal. Future studies should aim for a more The authors declare that they have no known competing financial
comprehensive quality coverage that includes characteristics not fully interests or personal relationships that could have appeared to influence
addressed by traditional standards, such as the ISO/IEC 25,000 series, the work reported in this paper.
Appendix A
1. Terms and definitions
To provide a better understanding of the application context in our SLR to the readers, we described the several key terms used in this work.
The terms are alphabetically ordered:
ADAS or Advanced Driver-Assistance System. These systems are focused on improving safety driving, offering features that can warn the driver
in case of misdriving (lane departure), distracted attention, or assume vehicle control (steering, braking, accelerating).
Approach. Based on the context of this SLR, we refer to the methodologies, processes, frameworks, models, methods, and techniques focused on
software data assessment phases.
Automotive SPICE (A-SPICE). It is a reference model for automotive process assessment [12].
Capability maturity model integration (CMMi). Is a bundle of practices to evaluate product development or services [12].
Characteristics impacts. The reported effects or influences on the software quality, such as safety, performance, user experience (UX), cost,
efficiency, and cyber-security.
Critical embedded systems (CES). The systems, in case of failures, can hardly impact the safety, security, and performance causing potential
damage to the environment or to human lives [15].
FuSa or Functional Safety. Is the “absence of unreasonable risks due to hazards caused by malfunctioning behaviour of Electric/Electronic
systems” [10]. ISO26262 is a standard that offers a framework to provide guidelines for automotive safety-related Electric/Electronic systems.
Property. In this work, the extracted data that contributes to research questions answering and/or aggregates value for the results analysis step.
Quality assessment characteristics. The main assessed attributes in the software (e.g. code quality, safety level, data accuracy). They were
extracted from selected studies.
Safety. Is a system attribute with the focus on software as the critical component [16].
Software data. In this work, “software data” means any software-related information that can be used for checking, measuring, comparing, or
evaluating via static (e.g., documentation, software code) or dynamic (e.g., test benches, vehicle, simulation) tests. The term software applied in this
work is based on the IEEE 1012–2016, which includes firmware and code. Software includes documentation.
Test maturity model integration (TMMi). Is a reference model for test process assessment and improvements [12].
User experience (UX). In the context of this SLR, is the impact created by a software quality improvement on the product in terms of facility, day-
by-day use, and efficiency perceived by the final customer [3,17,34,41].
Validation. This term is based on IEEE 1012–2016, which defines “validation” as the evaluation process of the software (as part of a system or
component) during or after the development process concerning the specified requirements. Validation processes include analysis, review, inspection,
assessment, and testing.
2. Categories description
The definitions were self-stated by the authors of the primary studies. Our classification was based on findings, and we did not intend to re-classify
them into other definition groups, rather the one stated in the primary studies. We acknowledge that overlaps or similarities exist among the different
terms and now discuss them in our Section 3.4-Threats to Validity and Limitations.
Approaches
We considered the self-stated approaches integrally.
• Techniques
• Methods
• Frameworks
• Models
• Processes
15
G. Pagoto et al. Computer Standards & Interfaces 97 (2026) 104110
• Case study
• Methodology
• Standard
• Systematic Mapping Study
• Topology
Application Focus / Target Groups
• Quality. When the studies present this general term, we understand that a bundle that was not described or split by the studies is behind. The
content of this bundle is detailed in standards, such as ISO 25,012, which support the critical characteristics for software quality evaluation.
• Safety. The term is covered by ISO 26,262 or only self-stated by studies.
• Performance. The combination of characteristics is covered by standards (ISO25012) and others, such as comfort.
• Cybersecurity. The term is covered by ISO 21,434, or only self-stated by studies.
• Traceability. The term is covered by ISO 25,012, or only self-stated by studies.
• Reliability. Term self-stated by studies.
Quality Assessment Application
• Validation. In same studies, the Validation is cited as Verification and Validation (V&V).
• Concept. Term self-stated by studies.
• Life Cycle. This classification defines the software phases from concept to decommissioning.
• Development Phases. The automotive V-Model establishes that “development phases” are Requirements & System Specification, Architecture &
Component Design, Coding or Implementation, and Tests (Component, Integration, System, and Acceptance).
• Field Applications. The end-user environment.
Product Type
• Embedded Software. The embedded software is an automotive software under evaluation and the focus of the studies’ assessment.
• Autonomous Systems. The Advanced Driver Assistance Systems are in scope. Examples: Active Brake Assist and Lane Keep Assist.
• Electronic Control Systems. The combination of embedded software and hardware.
• Powertrain. This term includes the engine, gearbox, and axles.
• Commercial Vehicles. Focused on vehicle-centered approaches.
• Connectivity. Cloud-based systems and others that use over-the-air solutions.
• Electric Vehicles. All categories of vehicles equipped with electric propulsion.
• Automation Systems. The automated gearbox is an example.
Software Data sources
• Specification. When the assessment data is based on specifications.
• Static/Dynamic Tests. When the Static/Dynamic tests provide reference data for other approaches.
• Source Code.
• Data Validation Set. When the storage data from tests is applied.
• Modelling. The result of modelling tasks is in focus.
• Historic Fault Data. When the storage fault data is applied.
• Simulation. The result of modelling tasks is in focus.
Reported Assessment Impacts
• Quality. When the studies present this general term, we understand that a bundle that was not described or split by the studies is behind. The
content of this bundle is detailed in standards, such as ISO 25,012, which support the critical characteristics for software quality evaluation.
• Safety. The term is covered by ISO 26,262 or only self-stated by studies.
• Performance. The combination of characteristics covered by standards (ISO25012) and others, such as comfort.
• User Experience (UX)
• Cost
• Efficiency. The term is covered by ISO 25,012 or is only self-stated by studies.
• Cybersecurity. The term is covered by ISO 21,434 or is only self-stated by studies.
3. Instructions for quality assessment criteria
We have created the instructions below to provide a better understanding of the quality assessment defined criteria.
Q1 - Is the study related to the proposed SLR objectives?
The target is to identify the presence of research questions topics in the studies.
• 1 point (Yes): The study contributes to all the research questions.
• 0,5 point (Partially): The study contributes at least one or more research question topic.
• 0 point (“No”): The study does not contribute to the research questions.
16
G. Pagoto et al. Computer Standards & Interfaces 97 (2026) 104110
Q2 - Does the study bring relevant information for answering research questions?
The target is to identify the detailing level of research question topics in the studies.
• 1 point (Yes): The study contributes to all the research questions.
• 0,5 point (Partially): The study contributes at least one or more research question topic.
• 0 point (“No”): The study does not contribute to the research questions.
Q3 - Is the quality application field clearly described and applicable to vehicle embedded software? (RQ1)
The target is to identify the description level of the quality approach field.
• 1 point (Yes): The study offers a dedicated section for application description.
• 0,5 point (Partially): The application description is present and can be identified in the text.
• 0 point (“No”): The study does not contribute to the research questions.
Q4 - Are the methodology, processes, models or framework clearly presented? (RQ1.1)
The target is to identify the description level of the adopted quality approaches.
• 1 point (Yes): The study offers a dedicated section for description.
• 0,5 point (Partially): The description is present and can be identified in the text.
• 0 point (“No”): The study does not contribute to the research questions.
Q5 - Is there any technique comparison included? If yes, is that clearly presented and described?
The target is to identify the presence and description level of the technique’s comparison.
• 1 point (Yes): The study offers a dedicated section for description.
• 0,5 point (Partially): The description is present and can be identified in the text.
• 0 point (“No”): The study does not contribute to the research questions.
Q6 - Does the study present any real use-case investigation, or is it applicable for? (RQ2)
The target is to identify the presence and description level of real use-cases.
• 1 point (Yes): The study offers a real use-case investigation and description.
• 0,5 point (Partially): The study presents example(s) of use-cases without detailing.
• 0 point (“No”): The study does not present real use-cases.
Q7 - Does the study bring consistent results regarding the presented technique(s) approach? (RQ3)
The target is to identify the presence and description level of the presented techniques.
• 1 point (Yes): The study offers results and description.
• 0,5 point (Partially): The study presents results without detailing.
• 0 point (“No”): The study does not present results.
Q8 - Does the study present success, unsuccess examples or issues during the software validation phase?
The target is to identify the presence and description level of critical results evaluation and challenges faced.
• 1 point (Yes): The study presents success, unsuccess, and challenges.
• 0,5 point (Partially): The study presents at least one of them.
• 0 point (“No”): The study does not present any results.
Q9 - Is the related study applicable to the commercial vehicles segment?
The target is to identify the applicability level for the commercial vehicles segment.
• 1 point (Yes): The study states the applicability for commercial vehicles.
• 0,5 point (Partially): The study presents general applicability coverage.
• 0 point (“No”): The study does not specify the applicability coverage.
Q10 - Does the study bring any coverage of quality issues related to software validation lacks?
The target is to identify the presence of examples of software validation lacks and their consequences.
• 1 point (Yes): The study presents a root cause relationship.
• 0,5 point (Partially): The study presents a general lack and/or consequences.
• 0 point (“No”): The study does not mention them.
17
G. Pagoto et al. Computer Standards & Interfaces 97 (2026) 104110
4-. Protocol application: candidate studies identification, screening process, and inclusion/exclusion results by database
Data availability International Conference on Range Technology (ICORT). IEEE, 2021, https://doi.
org/10.1109/ICORT52730.2021.9581624.
[12] E. Touw, Multi-faceted reliability assessment techniques: an industrial case study,
No data was used for the research described in the article. in: 2017 IEEE International Conference on Software Architecture Workshops
(ICSAW). IEEE, 2017, https://doi.org/10.1109/ICSAW.2017.66.
[13] T. Genevois, J.B. Horel, A. Renzaglia, et al., Augmented reality on lidar data: going
References
beyond vehicle-in-the-loop for automotive software validation, in: 2022 IEEE
Intelligent Vehicles Symposium (IV). IEEE, 2022, https://doi.org/10.1109/
[1] IEEE Standards Association, IEEE 1012-2016: IEEE standard for system, Software, IV51971.2022.9827351.
and hardware verification and validation (2017). [14] H. Kuwajima, H. Yasuoka, T. Nakae, Engineering problems in machine learning
[2] P.L. Goddard, Validating the safety of embedded real-time control systems using systems, Mach. Learn 109 (5) (2020) 1103–1126, https://doi.org/10.1007/
FMEA, in: Annual Reliability and Maintainability Symposium 1993 Proceedings. s10994-020-05872-w.
IEEE, 1993, https://doi.org/10.1109/RAMS.1993.296851. [15] D. Feitosa, A. Ampatzoglou, P. Avgeriou, et al., Design approaches for critical
[3] J. Zhang, Y. Lei, X. Hua, et al., Proposed shift quality metrics and experimentation embedded systems: a systematic mapping study, in: Evaluation of Novel
on AMT shift quality evaluation, in: Third International Conference on Natural Approaches to Software Engineering: 12th International Conference, ENASE 2017,
Computation (ICNC 2007) 2, 2007, https://doi.org/10.1109/ICNC.2007.582. Porto, Portugal, April 28–29, 2017.
IEEE. [16] P.V. Bhansali, Universal software safety standard, ACM SIGSOFT Softw. Eng. Notes
[4] L. Mäurer, T. Hebecker, T. Stolte, et al., On bringing object-oriented software 30 (5) (2005) 1–4, https://doi.org/10.1145/1095430.1095440.
metrics into the model-based world–Verifying ISO 26262 compliance in Simulink. [17] R. Rana, M. Staron, J. Hansson, et al., Defect prediction over software life cycle in
System analysis and modeling: models and reusability: 8th international automotive domain state of the art and road map for future, in: 2014 9th
conference, SAM 2014, Valencia, Spain, September 29-30, 2014. Proceedings 8, International Conference on Software Engineering and Applications (ICSOFT-EA).
Springer International Publishing, 2014, https://doi.org/10.1007/978-3-319- IEEE, 2014.
11743-0_15. [18] L.S. Myllyaho, et al., Systematic literature review of validation methods for AI
[5] M. Nyberg, D. Gurov, C. Lidström, et al., Formal verification in automotive systems, J. Syst. Softw. 181 (Nov. 2021) 111050, https://doi.org/10.1016/j.
industry: enablers and obstacles." leveraging applications of Formal methods, jss.2021.111050.
verification and validation. Industrial practice: 8th international symposium, ISoLA [19] B. Gezici, A.K. Tarhan, Systematic literature review on software quality for AI-
2018, Limassol, Cyprus, November 5-9, 2018, Proceedings, Part IV 8, Springer based software, Empir. Softw. Eng. 27 (3) (Mar. 2022), https://doi.org/10.1007/
International Publishing, 2018, https://doi.org/10.1007/978-3-030-03427-6_14. s10664-021-10105-2. Art. 66.
[6] I. Stürmer, E. Salecker, H. Pohlheim, Reviewing software models in compliance [20] M.A. Ali, et al., A systematic mapping of quality models for AI systems, software
with ISO 26262, in: Computer safety, reliability, and security: 31st international and components, Appl. Sci. 12 (17) (Aug. 2022), https://doi.org/10.3390/
conference, SAFECOMP 2012, Magdeburg, Germany, September 25-28, 2012. app12178700. Art. 8700.
Proceedings 31, Springer Berlin Heidelberg, 2012, https://doi.org/10.1007/978-3- [21] M. Krichen, Formal methods and validation techniques for ensuring automotive
642-33678-2_22. systems security, Information 14 (12) (Dec. 2023) 666, https://doi.org/10.3390/
[7] F. Wotawa, B. Peischl, F. Klück, et al., Quality assurance methodologies for info14120666.
automated driving, Elektrotech. Inf. 135 (4–5) (2018) 322–327, https://doi.org/ [22] B. Kitchenham, and S. Charters. "Guidelines for performing systematic literature
10.1007/s00502-018-0630-7. reviews in software engineering." 2007, 1051.
[8] ISO/IEC, ISO/IEC 25000: 2014 systems and software engineering-systems and [23] J. Biolchini, P.G. Mian, A.C.C. Natali, et al., Systematic review in software
software Quality Requirements and Evaluation (SQuaRE)-Guide to SQuaRE, Inf. engineering, Syst. Eng. Comput. Sci. Dep. COPPE/UFRJ Tech. Rep. ES 679.05
technol. stand. (2014). (2005) 45.
[9] International Organization for Standardization, ISO/SAE 21434: 2021: Road [24] V. Parsifal, 2021. Available online: https://parsif.al (accessed on 24.03.2024).
Vehicles: Cybersecurity Engineering, ISO, (2021). [25] D. Stefanovic, S. Havzi, D. Nikolic, et al., Analysis of the tools to support systematic
[10] International Organization for Standardization (ISO). ISO 26262-1: 2018–road literature review in software engineering. IOP conference series: materials science
vehicles—functional safety. Geneva, Switzerland. 2018. and engineering. Vol. 1163. No. 1, IOP Publishing, 2021.
[11] P. Kumar, L.K. Singh, C. Kumar, et al., A Bayesian belief network model for early [26] C. Wohlin, P. Runeson, M. Höst, et al., Experimentation in software engineering,
prediction of reliability for computer-based safety-critical systems, in: 2021 2nd Springer Science & Business Media, 2012.
18
G. Pagoto et al. Computer Standards & Interfaces 97 (2026) 104110
[27] Kitchenham, B.A., Budgen, D., & Brereton, P. “Evidence-based software [45] E. Bringmann, A. Krämer, Model-based testing of automotive systems, in: 2008 1st
engineering and systematic reviews.” CRC press., 2015. international conference on software testing, verification, and validation. IEEE,
[28] J.R. Landis, G.G. Koch, An application of hierarchical kappa-type statistics in the 2008, https://doi.org/10.1109/ICST.2008.45.
assessment of majority agreement among multiple observers, Biometrics (1977) [46] C. Kugler, S. Kowalewski, J. Richenhagen, et al., Metrics-based strategies for
363–374. quality assurance of automotive embedded software. 17. Internationales stuttgarter
[29] I. Sommerville, Software engineering, 9"‘ed, Engl.: Educ. Ltd. (2010). symposium: Automobil-und Motorentechnik, Springer, Fachmedien Wiesbaden,
[30] J. Saldaña, The coding manual for qualitative researchers, Sage (2015). 2017, https://doi.org/10.1007/978-3-658-16988-6_56.
[31] D.M. Bailey, J.M. Jackson, Qualitative data analysis: challenges and dilemmas [47] J. Jürjens, D. Reiß, D. Trachtenherz, Model-based quality assurance of automotive
related to theory and method, Am. J. Occup. Ther. 57 (1) (2003) 57–65. Publisher: software. International conference on model driven engineering languages and
American Occupational Therapy Association. systems, Springer Berlin Heidelberg, Berlin, Heidelberg, 2008, https://doi.org/
[32] M. Takrouni, M. Gdhaifi, A. Hasnaoui, et al., Design and implementation of a 10.1007/978-3-540-87875-9_59.
simulink DDS blockset and its integration to an active frame steering blockset [48] M. Ellims, R. Evans, K.M. Hobley, et al., When is software ready for production?
conformed to SAE ElectricVehicle, in: 2017 IEEE/ACS 14th International Parallels with automotive QS9000 methods, in: Aspects of safety management:
Conference on Computer Systems and Applications (AICCSA). IEEE, 2017, https:// proceedings of the ninth safety-critical systems symposium, Bristol, UK 2001,
doi.org/10.1109/AICCSA.2017.52. Springer London, London, 2000, https://doi.org/10.1007/978-1-4471-0713-2_9.
[33] A. Mateen, Q. Zhu, S. Afsar, Comparitive analysis of manual vs automotive testing [49] X. Li, W.E. Wong, R. Gao, et al., Genetic algorithm-based test generation for
for software quality, in: Proceedings of the 7th International Conference on software product line with the integration of fault localization techniques, Empir.
Software Engineering and New Technologies, 2018, https://doi.org/10.1145/ Softw. Eng. 23 (2018) 1–51, https://doi.org/10.1007/s10664-016-9494-9.
3330089.3330121. [50] D.K. Oka, T. Makila, R. Kuipers, Integrating application security testing tools into
[34] X. Lu, H. Chen, B. Gao, et al., Data-driven predictive gearshift control for dual- ALM tools in the automotive industry, in: 2019 IEEE 19th International Conference
clutch transmissions and FPGA implementation, IEEE Trans. Ind. Electron. 62.1 on Software Quality, Reliability and Security Companion (QRS-C). IEEE, 2019,
(2014) 599–610, https://doi.org/10.1109/TIE.2014.2312312. https://doi.org/10.1109/QRS-C.2019.00021.
[35] K. Ranasinghe, R. Sabatine, A. Gardi, et al., Advances in integrated system health [51] O. Kovalenko, E. Serral, M. Sabou, et al., Automating cross-disciplinary defect
management for mission-essential and safety-critical aerospace applications, Prog. detection in multi-disciplinary engineering environments, Knowledge engineering
Aerosp. Sci. 128 (2022), https://doi.org/10.1016/j.paerosci.2021.100758. and knowledge management: 19th international conference, EKAW 2014,
[36] M.K. Thota, F.H. Shajin, P. Rajesh, Survey on software defect prediction Linköping, Sweden, November 24-28, 2014. Proceedings 19 (2014), https://doi.
techniques, Int. J. Appl. Sci. Eng. 17 (4) (2020) 331–344, https://doi.org/10.6703/ org/10.1007/978-3-319-13704-9_19.
IJASE.202012_17(4).331. [52] J.R.V. Rivero, T. Gerbich, B. Buschardt, et al., The effect of spray water on an
[37] H. Ebadi, M.H. Moghadam, M. Borg, et al., Efficient and effective generation of test automotive LIDAR sensor: a real-time simulation study, IEEE Trans. Intell. Veh. 7.1
cases for pedestrian detection-search-based software testing of Baidu Apollo in (2021) 57–72, https://doi.org/10.1109/TIV.2021.3067892.
SVL, in: 2021 IEEE International Conference on Artificial Intelligence Testing [53] L.J. Moukahal, M. Zulkernine, M. Soukup, Vulnerability-oriented fuzz testing for
(AITest). IEEE, 2021, https://doi.org/10.1109/AITEST52744.2021.00030. connected autonomous vehicle systems, IEEE Trans. Reliab. 70 (4) (2021)
[38] H. Schweppe, A. Zimmermann, D. Grilly, Flexible in-vehicle stream processing 1422–1437, https://doi.org/10.1109/TR.2021.3112538.
with distributed automotive control units for engineering and diagnosis, in: 2008 [54] E.Y. Kang, D. Mu, L. Huang, et al., Verification and validation of a cyber-physical
International Symposium on Industrial Embedded Systems. IEEE, 2008, https:// system in the automotive domain, in: 2017 IEEE International Conference on
doi.org/10.1109/SIES.2008.4577683. Software Quality, Reliability and Security Companion (QRS-C). IEEE, 2017,
[39] O. Sadio, I. Ngom, C. Lishou, Design and prototyping of a software defined https://doi.org/10.1109/QRS-C.2017.62.
vehicular networking, IEEE Trans. Veh. Technol. 69 (1) (2019) 842–850, https:// [55] M. Mauritz, F. Howar, A. Rausch, Assuring the safety of advanced driver assistance
doi.org/10.1109/TVT.2019.2950426. systems through a combination of simulation and runtime monitoring. Leveraging
[40] M. Kläs, T. Bauer, A. Dereani, et al., A large-scale technology evaluation study: applications of formal methods, verification and validation: discussion,
effects of model-based analysis and testing, in: 2015 IEEE/ACM 37th IEEE dissemination, applications: 7th international symposium, ISoLA 2016, Imperial,
International Conference on Software Engineering. Vol. 2. IEEE, 2015, https://doi. Corfu, Greece, October 10-14, 2016, Proceedings, Part II 7, Springer International
org/10.1109/ICSE.2015.141. Publishing, 2016, https://doi.org/10.1007/978-3-319-47169-3_52.
[41] P.J. Kollmeyer, M. Naguib, F. Khanum, et al., A blind modeling tool for [56] P. Skruch, M. Długosz, P. Markiewicz, A formal approach for the verification of
standardized evaluation of battery state of charge estimation algorithms, in: 2022 control systems in autonomous driving applications, in: Trends in Advanced
IEEE Transportation Electrification Conference & Expo (ITEC). IEEE, 2022, https:// Intelligent Control, optimization and automation: proceedings of KKA 2017—the
doi.org/10.1109/ITEC53557.2022.9813996. 19th Polish control conference, Kraków, Poland, June 18–21, 2017, Springer
[42] N. Kovačić, V. Ilić, A. Korać, et al., Validation of process execution on automotive International Publishing, 2017, https://doi.org/10.1007/978-3-319-60699-6_18.
embedded devices, in: 2017 13th International Conference on Advanced [57] K. Liu, W. Kong, G. Hou, et al., A survey of formal techniques for hardware/
Technologies, Systems and Services in Telecommunications (TELSIKS). IEEE, 2017, software co-verification, in: 2018 7th International Congress on Advanced Applied
https://doi.org/10.1109/TELSKS.2017.8246318. Informatics (IIAI-AAI). IEEE, 2018, https://doi.org/10.1109/IIAI-AAI.2018.00033.
[43] R. Jiang, Required characteristics for software reliability growth models. 2009 WRI [58] N. Mellegård, M. Staron, F. Törner, A light-weight defect classification scheme for
world congress on software engineering. Vol. 4, IEEE, 2009, https://doi.org/ embedded automotive software and its initial evaluation, in: 2012 IEEE 23rd
10.1109/WCSE.2009.157. International Symposium on Software Reliability Engineering. IEEE, 2012, https://
[44] P. Herber, S. Glesner, Verification of embedded real-time systems, in: Formal doi.org/10.1109/ISSRE.2012.15.
Modeling and Verification of Cyber-Physical Systems: 1st International Summer [59] D.K. Kim, J.H. Kim, M.C. Lee, Improvement of HILS using advanced exploratory
School on Methods and Tools for the Design of Digital Systems, Bremen, Germany and optimization techniques for system qualification test, Int. J. Automot. Technol.
2015, September 2015, pp. 1–25, https://doi.org/10.1007/978-3-658-09994-7_1. 24 (3) (2023) 901–911, https://doi.org/10.1007/s12239-023-0074-x.
19
Reference in New Issue View Git Blame Copy Permalink