How I trained my first CNN to detect COVID-19 in chest X-rays

I had never trained a neural network for images before. This is my log of training my first CNN to distinguish normal chest X-rays from COVID-19 cases: what confused me, what finally clicked and the mistakes that taught me more than the successes.

01 What is a CNN, and why use one for images?

The first thing I had to understand was that a standard neural network sees an image as a huge list of disconnected pixels and loses the idea of what is next to what. A CNN (convolutional neural network) solves this through convolution: a small window slides across the image looking for patterns such as edges, textures and shapes.

The image that unlocked the intuition for me was a magnifying glass moving across the X-ray and learning by itself what to inspect. Early layers detect simple things such as lines and contrasts; later layers combine them into more complex structures such as lung shapes and opacities. No one tells it what to look for; it learns from examples.

02 Preparing the data was 70% of the work

I thought the difficult part would be “the network.” That was wrong: preparing the images took most of the time. The dataset had X-rays in two folders (COVID and normal), and before training I had to:

• Resize everything to 224×224 pixels, because the network needs every image to have the same dimensions.
• Normalize the pixels from 0–255 to a range of 0–1 by dividing by 255. This helps the network train more reliably.
• Augment the data (data augmentation): rotate, zoom and flip the images to create variations. With a limited number of medical images, this technique is extremely valuable.
• Split the available training data into 80% training and 20% validation, while keeping the test set separate for the final evaluation. This prevents me from evaluating the model on the same images used for training.

from tensorflow.keras.preprocessing.image import ImageDataGenerator

train_datagen = ImageDataGenerator(
    rescale=1./255,        # from 0–255 to 0–1
    rotation_range=15,     # rotate up to ±15°
    zoom_range=0.1,        # 10% zoom
    horizontal_flip=True,   # horizontal flip
    validation_split=0.2
)

03 Building the network in blocks—it was not so mystical

Thinking of it as repeating blocks made it less intimidating. I used three convolutional blocks, each following the same recipe: a layer that searches for patterns (Conv2D), one that stabilizes (BatchNormalization), one that reduces dimensions (MaxPooling) and one that randomly deactivates neurons so the model does not memorize (Dropout).

model = models.Sequential([
    # Block 1: simple patterns
    layers.Conv2D(32, (3,3), activation='relu', padding='same'),
    layers.BatchNormalization(),
    layers.MaxPooling2D(2,2),
    layers.Dropout(0.25),
    # Blocks 2 and 3: more filters (64 and 128), more complex patterns
    # ...
    layers.Flatten(),
    layers.Dense(256, activation='relu'),
    layers.Dropout(0.5),
    layers.Dense(1, activation='sigmoid')  # 1 output: COVID or normal
])

The final layer has a single neuron with sigmoid, which returns a number between zero and one: the “probability” that the X-ray is COVID. If it is greater than 0.5, the model classifies it as COVID.

04 The specter of overfitting

The concept I struggled with most was overfitting: when the model memorizes the images instead of learning the general pattern. It performs perfectly on what it has already seen and poorly on new images. It is like a student who memorizes the answers to a practice test without understanding the subject.

I used three tools against it, and seeing them work was very satisfying:

• Dropout: randomly deactivating neurons forces the network not to depend on a single path.
• Data augmentation: seeing the same image rotated or zoomed prevents the network from simply memorizing it.
• EarlyStopping: stops training automatically when the model stops improving, instead of continuing until it memorizes the data.

In the end, the difference between training and validation performance stayed below 3%. That signal—similar performance on unseen data—told me that the model was genuinely learning.

05 Metrics: why accuracy is not enough

My first instinct was to look only at accuracy: the percentage of correct predictions. I reached 92.86% and felt delighted. But in a medical problem, that number can be misleading, and this became the project’s most important lesson.

Imagine that 90 out of every 100 X-rays are normal. A dishonest model that always predicts “normal” would achieve 90% accuracy while missing every COVID case. That is why other metrics matter:

• Recall (sensitivity): out of all real COVID cases, how many did I detect? I reached 94.29%. In medicine this is crucial, because the most serious error is a false negative (classifying a sick person as healthy).
• Precision: out of the cases labeled COVID, how many really were COVID? (91.43%).
• F1 score: a balance between the previous two metrics (0.928).

06 What I found most difficult—honestly

For this to be a genuine log rather than a perfect snapshot, these are the parts I found most difficult:

• Dimension errors. I spent hours fighting shape mismatch. In the end, it was almost always an image that had not been resized correctly or an incorrect input size.
• Trusting accuracy. It was difficult to accept that 92% could be misleading. Changing the question to “Which error is more costly?” was the most valuable lesson of the entire project.
• Invisible overfitting. I did not understand why the model performed brilliantly during training and poorly on new images. Seeing the training and validation curves diverge was the moment it clicked.
• Patience. Training takes time, and watching the numbers change can be anxiety-inducing. I learned to let EarlyStopping make the decision instead of stopping training myself.
• Not copying and pasting without understanding. It is easy to paste code from a tutorial and make it “work.” Forcing myself to understand what each layer did was slower, but it was what made the learning real.

07 What I took away

Training my first CNN taught me that deep learning is not black magic. It is about preparing the data properly, building the network from meaningful blocks, controlling overfitting and selecting metrics according to the problem. Every step has a rationale, and once you understand it, the process becomes far less intimidating.

The next step? Trying Transfer Learning: instead of training from scratch, start with a network already trained on millions of images, such as ResNet or MobileNet, and adapt it. It is said to perform better with limited data, and I want to test that myself.