Improving efficiency of diesel engine fault detection based on multi-source data

2.1 Data

To comprehensively record the actual operating conditions of various equipment in the engine room, sensors were installed on a 9,196-ton training ship, with its specifications detailed in Table 1. The data was collected through the ship’s VIMS system (Table 1) during a full day of training voyage on June 25, 2022, resulting in 8,420 data points.

Table 1:

Main specifications of the training ship

The collected data features are categorized into three types: operational features, which include speed, main engine propeller pitch feedback, and main engine RPM, directly influenced by navigational operations and reflecting the vessel’s running state—these parameters help distinguish between normal operational variations and abnormal mechanical failures in fault detection; global features, which describe the overall performance of the main engine, including fuel supply, cooling system, and scavenging system—these features represent the overall health of the main engine rather than a specific component; and target features, which are the core indicators primarily used for fault detection, playing a crucial role in identifying anomalies and potential failures.

The complex operating environment inside the engine room may affect the reliability of sensors, leading to data loss or noise interference [3]. Therefore, effective data cleaning is essential to ensure the accuracy and reliability of sensor data. This study first classifies the voyage states to distinguish the characteristics of data under different operating conditions. When the ship’s acceleration exceeds 0.2 knots per minute, it is defined as an unstable navigation state. Subsequently, data cleaning is performed separately for each voyage state [18]. The μ±3σ rule is applied to determine the upper and lower limits of each feature value, and outliers beyond this range are removed, where μ represents the mean and σ denotes the standard deviation. For the removed outliers and originally missing data, the mean value of the adjacent data points is used for imputation. If too many consecutive data points are missing, a smoothing interpolation method is applied based on the number of missing values to reduce the impact of data fluctuations on the analysis.

Due to the different scales of each feature, and with the aim to reduce the impact of scale inconsistency on the model and decrease computational complexity, this study applies the Min-Max Normalization method to standardize the data, mapping the values to the range of 0 to 1. The algorithm is demonstrated in Equation (1),

where $$ X_t^{\prime }$$ is the new value after normalization, X_tis the original value to be normalized, X_tmax is the maximum value of the data sample, and X_tmin is the minimum value of the data sample.

2.2 Models

2.2.1 RNN

RNN is a type of neural network designed for processing sequential data. Its core characteristic lies in the use of a recurrent structure (Figure 1), allowing computations at the current time step to depend on information from previous time steps. The RNN maintains a hidden state to store information from the previous time step (Equation (2)) and passes it to the next time step, thereby capturing long-term dependencies. The output at the current time step is determined by both the hidden state from the previous time step and the current input (Equation (3)).

$\[ S_t=f\left(U{X}_t+W{S}_{t-1}+b\right) \]$

(2)

$\[ O_t=g\left(V{S}_t\right) \]$

(3)

Figure 1:

Structure of RNN

Here, t represents the current time step, S denotes the hidden state, and X refers to the input information. U is the weight matrix for the input, while W is the weight matrix for the hidden layer. f represents the activation function, V is the weight matrix from the hidden layer to the output layer, and g is the output activation function. Additionally, b denotes the bias term.

2.2.2 LSTM

In an RNN structure, as information propagates through the network, the influence of early inputs gradually diminishes, potentially leading to the loss of long-term dependencies. To address this issue, the LSTM network was introduced. As a specialized type of recurrent neural network, LSTM is designed to capture long-term dependencies. Compared to traditional RNN, LSTM incorporates a gating mechanism that effectively mitigates the vanishing gradient problem, enhancing stability and accuracy when processing long sequences (Figure 2).

Figure 2:

Structure of LSTM

The forget gate determines how much of the current unit’s memory state should be discarded (Equation (4)).

$\[ f_t=\sigma \left(W_f\bullet \left[h_{t-1},x_t\right]+b_f\right) \]$

(4)

Where f_t is the output of the forget gate, h_t-1 is the hidden state from the previous time step, x_t is the input at the current time step, W_f​ and b_f​​ are the trainable weights and biases, and σ is the Sigmoid activation function.

The input gate determines which parts of the current input will be used to update the cell state (Equation (5)), and the candidate memory decides how the cell state is updated at the current time step (Equation (6)).

$\[ i_t=\sigma \left(W_i\bullet \left[h_{t-1},x_t\right]+b_i\right) \]$

(5)

Where i_t is the output of the input gate, which functions similarly to the forget gate, W_i and b_i are the weights and biases.

$\[ {\tilde{C}}_t=tan h\left(W_c\bullet \left[h_{t-1}, x_t\right]+b_c\right) \]$

(6)

Where $$ {\tilde{C}}_t$$ is the candidate cell state, and tan h is the hyperbolic tangent activation function. W_c and b_c are the weights and biases.

The current cell state is then computed by updating the memory from the previous time step's cell state, incorporating the memory portion controlled by the forget and input gates (Equation (7)).

$\[ C_t=f_t\bullet C_{t-1}+i_t\bullet {\tilde{C}}_t \]$

(7)

Where C_t is the current cell state.

The output gate determines the content of the hidden state h_t at the current time step, and the final hidden state h_t is computed by combining the current cell state C_t and the output gate O_t (Equation (8), (9)).

$\[ O_t=\sigma \left(W_0\bullet \left[h_{t-1},x_t\right]+b_0\right) \]$

(8)

$\[ h_t=O_t\bullet tanh\left(C_t\right) \]$

(9)

2.2.3 GRU

GRU is a simplified version of the RNN, designed to address the vanishing gradient problem. Compared to LSTM, the GRU has a more streamlined structure, offering higher computational efficiency in some applications (Figure 3). It combines the input and forgets gates of LSTM into a single update gate and uses a reset gate to control the focus on past information, effectively capturing long-term dependencies in sequential data.

Figure 3:

Structure of GRU

The reset gate determines how much information should be copied from the hidden state of the previous time step. As calculated in Eq. 10,

$\[ r_t=\sigma \left(W_r\bullet \left[h_{t-1},x_t\right]+b_r\right) \]$

(10)

where r_t is the reset gate vector, W_r is the weight matrix for x_t, h_t-1 is the hidden state from the previous time step, x_t is the input, σ is the Sigmoid activation function, and b_r is the bias term.

The update gate determines how much information from the previous hidden state should be incorporated into the current hidden state (Equation (11)). Then, the candidate hidden state is computed (Equation (12)), and the new hidden state is calculated based on the combination of the candidate hidden state and the update gate (Equation (13)).

$\[ z_t=\sigma \left(W_z\bullet \left[h_{t-1},x_t\right]+b_z\right) \]$

(11)

$\[ {\tilde{h}}_t=tan h\left(W_h\bullet \left[r_t\bullet h_{t-1},x_t\right]+b_t\right) \]$

(12)

$\[ h_t=\left(1-z_t\right)\bullet h_{t-1}+z_t\bullet {\tilde{h}}_t \]$

(13)

In Eq.11, z_t is the update gate vector, $$ {\tilde{h}}_t$$ in Equation (12) is the candidate hidden state, W_z in Equation (11) and W_h in Equation (12) are the weight matrices for x_t, b_z in Equation (11) and b_t in Equation (12) are the bias terms, and tan h in Equation (12) is the activation function. In Equation (13), 1 - z_t controls the proportion of new information introduced.

2.3 Model Evaluation

This paper uses R², Mean Absolute Error (MAE), Mean Squared Error (MSE), and Mean Absolute Percentage Error (MAPE) to evaluate the model, aiming to comprehensively measure the model’s predictive ability and error characteristics.

R² is a standard metric for evaluating the fit of a regression model, representing the proportion of the variance in the target features that the model can explain (Equation (17)). The value of R² ranges from 0 to 1, with values closer to 1 indicating better model fit.

where SS_res is the sum of squared residuals, SS_tot is the total sum of squares, y_i is the observed value, $$ {\widehat{y}}_i$$ is the predicted value, and $$ \stackrel{-}{y}$$ is the mean of the observed values.

MAE measures the average absolute difference between the model’s predicted values and the actual values, offering intuitive interpretability (Equation (18)). A smaller MAE indicates higher model accuracy.

MSE evaluates the model’s predictive ability by calculating the square of the prediction errors (Equation (19)). MSE imposes a stronger penalty for larger errors, making it more sensitive to the model’s performance when there are significant deviations. A smaller MSE indicates higher model accuracy.

MAPE is a relative error metric that measures the percentage of prediction error relative to the actual value (Equation (20)). This metric is particularly useful for comparing data of different scales, providing an intuitive perception of the model’s relative accuracy. A smaller MAPE indicates higher model accuracy.

3.1 Development of a Predictive Model

This study explores whether combining parameter manipulation with global engine characteristics for joint modeling can improve the prediction accuracy of target features. To achieve this, we constructed models using three deep learning algorithms—RNN, LSTM, and GRU—and evaluated their performance using four assessment metrics. The evaluation was conducted exclusively on the target features.

According to the results in Tables 2 and 3, the highest R² was observed in the LSTM model built solely on target features, while the lowest MAE and MAPE were achieved by the GRU model using only target features. Meanwhile, the lowest MSE was obtained from the RNN and LSTM models built using all features.

Table 2:

Evaluation metrics for modeling based on all features

Table 3:

Evaluation metrics for modeling based on target features

Comparing the average values of the two modeling approaches (Table 4), the results indicate that, overall, modeling with all features yields better performance, which contradicts the sources of some individual optimal metrics. We believe this discrepancy is primarily due to the RNN model built solely on target features, whose evaluation metrics exhibited significant deviations, thereby affecting the overall average results. Consequently, the current analysis still supports modeling using only target features.

Table 4:

Mean score of evaluation metrics for each modeling approach

This result contradicts our initial expectations. Further analysis of the fitting performance of each feature (Figure 4) revealed that some global features exhibited poor fit, potentially failing to enhance the predictive capability of the target features. Therefore, in the following section, these features were removed during modeling, and the model was reconstructed to optimize prediction performance.

Figure 4:

Cases of partially unfit features

3.2 Modeling with All Features Excluding Unfitted Ones

As in Figure 4, the M/E F.O inlet pressure and the M/E JCFW inlet pressure cannot be fitted in the model. The R² of the main engine speed and the F.O inlet temperature are also too low. Likewise, as shown in this figure, the errors of other features are also relatively large. Such global features cannot bring help to the prediction of the whole model. Therefore, these six features were deleted and excluded from the training data. The modeling is renewed using the three algorithms.

The evaluation metrics for the three models are shown in Table 5. Compared to modeling with all features or using only target features, the RNN model achieved the highest R² and performed best in terms of MAE and MSE. Meanwhile, the GRU model obtained the lowest MAPE, and its performance was similar to that of the RNN model.

Table 5:

Evaluation metrics for models excluding unfitted features

A comparison of the average evaluation metrics across the three modeling approaches (Table 6) revealed that all optimal metrics were obtained from the modeling approach that excluded the poorly fitted features. Ultimately, we confirmed that incorporating selected data reflecting the overall operational state of the engine room, along with control data, can effectively improve the prediction accuracy of target features.

Table 6:

Mean score of three modeling approaches to three evaluating indicators

3.3 Modeling with All Features Excluding Unfitted Ones

The maritime industry is generally reluctant to share performance and status datasets, particularly when the data may contain errors. As a result, researchers typically use simulated faults to test models, employing linear adjustment and expansion methods to generate out-of-limit fault data [19].

When constructing the EWMA control chart, it is necessary to continuously predict the target value using residuals. Due to the model’s construction approach, predictions require a combined dataset of engine room data, operational data, and target data. However, simply using linear adjustments and expansions of target features does not yield the corresponding engine room and manipulation data, thus preventing the use of this method for model validation.

A literature review reveals that no study has explicitly explored the relationship between prediction accuracy and EWMA control charts. Therefore, this study developed a pure mathematical model and analyzed the relationship between prediction accuracy and EWMA through simulations of normal and fault data. For this, 100 arrays with an average value of 10 were created, and random small fluctuations were added to represent the true values. The predicted values were set with varying fluctuations depending on the accuracy and stability (Table 7).

Table 7:

Evaluation of mathematical simulations of predicted and real values

As revealed in Table 7, the values for low accuracy and high stability were similar to those for high accuracy and high stability. This is because when accuracy is low, stability is necessarily poor, and when stability is poor, accuracy is also low. Therefore, numerical simulation can only ensure a relative ranking of both accuracy and stability.

Figure 5 shows the generated arrays, and the EWMA control lines based on these data are shown in Figure 6. It can be observed that higher accuracy and stability result in smaller EWMA control limits, making it easier to detect faults. It indicated that when the predictive model shows accurate and stable performance, the residuals tend to become smaller and accordingly the EWMA values and control limits are also smaller. When a fault occurs, the predicted value will deviate from the expected values and the residuals will change significantly, making them easier to be recognized by the EWMA.

Figure 5:

True Values and Predictions in Different Scenarios

Figure 6:

EWMA control limit in four scenarios

Additionally, the control limits for high accuracy and low stability are slightly smaller than those for low accuracy and high stability. This further indicates that when accuracy and stability are similar, higher accuracy facilitates fault detection.

To test the fault detection capability of the model, we added 50 sets of data with a mean value of 13 and introduced random small fluctuations. Meanwhile, the mean of the predicted values remained at 10, indicating that the model failed to correctly predict the true values. As shown in Fig. 7, the EWMA control chart with high accuracy and high stability detected faults the earliest. In contrast, as for the EWMA control chart with low accuracy and low stability, although from position 142 some points of EWMA value exceeded the control limits, the EWMA values was observed to quickly return within the control limits. Additionally, the data with high accuracy but low stability exceeded the control limits earlier than the data with low accuracy but high stability, but both sets of data continued to fluctuate both inside and outside the control limits.

Figure 7:

EWMA control charts

The final conclusion is that under conditions of high accuracy and high stability, the EWMA control limits are smaller, making fault detection easier. Furthermore, during the early stages of a fault, the data may fluctuate repeatedly inside and outside the control limits. Therefore, a threshold should be set to determine whether the duration of continuous exceedance of the control limits is sufficient to confirm the occurrence of a fault.

Author Contributions

Conceptualization, S. Piao and W. J. Lee; Methodology, S. Piao; Writing – Original Draft & Editing, S. Piao; Data Collection & Curation, J. J. Hur, J. H. Im, and M. K. Kang; Supervision, W. J. Lee; Project Administration, W. J. Lee; Writing – Review & Editing, S. Piao and W. J. Lee..