Picture Augmentation for Deep Studying with Keras


Final Up to date on August 6, 2022

Knowledge preparation is required when working with neural networks and deep studying fashions. More and more, information augmentation can also be required on extra advanced object recognition duties.

On this submit, you’ll uncover the right way to use information preparation and information augmentation along with your picture datasets when creating and evaluating deep studying fashions in Python with Keras.

After studying this submit, you’ll know:

  • In regards to the picture augmentation API supplied by Keras and the right way to use it along with your fashions
  • carry out function standardization
  • carry out ZCA whitening of your photos
  • increase information with random rotations, shifts, and flips
  • save augmented picture information to disk

Kick-start your undertaking with my new guide Deep Studying With Python, together with step-by-step tutorials and the Python supply code recordsdata for all examples.

Let’s get began.

  • Jun/2016: First revealed
  • Replace Aug/2016: The examples on this submit had been up to date for the newest Keras API. The datagen.subsequent() operate was eliminated
  • Replace Oct/2016: Up to date for Keras 1.1.0, TensorFlow 0.10.0 and scikit-learn v0.18
  • Replace Jan/2017: Up to date for Keras 1.2.0 and TensorFlow 0.12.1
  • Replace Mar/2017: Up to date for Keras 2.0.2, TensorFlow 1.0.1 and Theano 0.9.0
  • Replace Sep/2019: Up to date for Keras 2.2.5 API
  • Replace Jul/2022: Up to date for TensorFlow 2.x API with a workaround on the function standardization subject

For an prolonged tutorial on the ImageDataGenerator for picture information augmentation, see:

Keras Picture Augmentation API

Like the remainder of Keras, the picture augmentation API is straightforward and highly effective.

Keras supplies the ImageDataGenerator class that defines the configuration for picture information preparation and augmentation. This consists of capabilities corresponding to:

  • Pattern-wise standardization
  • Characteristic-wise standardization
  • ZCA whitening
  • Random rotation, shifts, shear, and flips
  • Dimension reordering
  • Save augmented photos to disk

An augmented picture generator may be created as follows:

Relatively than performing the operations in your whole picture dataset in reminiscence, the API is designed to be iterated by the deep studying mannequin becoming course of, creating augmented picture information for you simply in time. This reduces your reminiscence overhead however provides some extra time price throughout mannequin coaching.

After you might have created and configured your ImageDataGenerator, you need to match it in your information. It will calculate any statistics required to truly carry out the transforms to your picture information. You are able to do this by calling the match() operate on the info generator and passing it to your coaching dataset.

The information generator itself is, in reality, an iterator, returning batches of picture samples when requested. You’ll be able to configure the batch measurement and put together the info generator and get batches of photos by calling the circulation() operate.

Lastly, you can also make use of the info generator. As a substitute of calling the match() operate in your mannequin, you need to name the fit_generator() operate and go within the information generator and the specified size of an epoch in addition to the whole variety of epochs on which to coach.

You’ll be able to study extra in regards to the Keras picture information generator API within the Keras documentation.

Need assistance with Deep Studying in Python?

Take my free 2-week electronic mail course and uncover MLPs, CNNs and LSTMs (with code).

Click on to sign-up now and in addition get a free PDF E-book model of the course.

Level of Comparability for Picture Augmentation

Now that you understand how the picture augmentation API in Keras works, let’s have a look at some examples.

We are going to use the MNIST handwritten digit recognition job in these examples. To start with, let’s check out the primary 9 photos within the coaching dataset.

Working this instance supplies the next picture that you should utilize as a degree of comparability with the picture preparation and augmentation within the examples beneath.

Instance MNIST photos

Characteristic Standardization

It is usually doable to standardize pixel values throughout your entire dataset. That is referred to as function standardization and mirrors the kind of standardization usually carried out for every column in a tabular dataset.

You’ll be able to carry out function standardization by setting the featurewise_center and featurewise_std_normalization arguments to True on the ImageDataGenerator class. These are set to False by default. Nonetheless, the latest model of Keras has a bug within the function standardization in order that the imply and customary deviation is calculated throughout all pixels. When you use the match() operate from the ImageDataGenerator class, you will note a picture much like the one above:

For instance, the minimal, imply, and most values from the batch printed above are:

And the picture displayed is as follows:

Picture from feature-wise standardization

The workaround is to compute the function standardization manually. Every pixel ought to have a separate imply and customary deviation, and it must be computed throughout totally different samples however unbiased from different pixels in the identical pattern. You simply want to interchange the match() operate with your personal computation:

The minimal, imply, and most as printed now have a wider vary:

Working this instance, you may see that the impact is totally different, seemingly darkening and lightening totally different digits.

Standardized function MNIST photos

ZCA Whitening

A whitening rework of a picture is a linear algebraic operation that reduces the redundancy within the matrix of pixel photos.

Much less redundancy within the picture is meant to higher spotlight the constructions and options within the picture to the educational algorithm.

Usually, picture whitening is carried out utilizing the Principal Part Evaluation (PCA) method. Extra just lately, an alternate referred to as ZCA (study extra in Appendix A of this tech report) exhibits higher leads to remodeled photos that maintain all the unique dimensions. And in contrast to PCA, the ensuing remodeled photos nonetheless appear to be their originals. Exactly, whitening converts every picture right into a white noise vector, i.e., every aspect within the vector has zero imply and unit customary derivation and is statistically unbiased of one another.

You’ll be able to carry out a ZCA whitening rework by setting the zca_whitening argument to True. However as a result of identical subject as function standardization, you need to first zero-center your enter information individually:

Working the instance, you may see the identical basic construction within the photos and the way the define of every digit has been highlighted.

ZCA whitening MNIST photos

Random Rotations

Generally photos in your pattern information could have various and totally different rotations within the scene.

You’ll be able to practice your mannequin to higher deal with rotations of photos by artificially and randomly rotating photos out of your dataset throughout coaching.

The instance beneath creates random rotations of the MNIST digits as much as 90 levels by setting the rotation_range argument.

Working the instance, you may see that photos have been rotated left and proper as much as a restrict of 90 levels. This isn’t useful on this drawback as a result of the MNIST digits have a normalized orientation, however this rework may be of assist when studying from images the place the objects could have totally different orientations.

Random rotations of MNIST photos

Random Shifts

Objects in your photos might not be centered within the body. They could be off-center in a wide range of other ways.

You’ll be able to practice your deep studying community to anticipate and presently deal with off-center objects by artificially creating shifted variations of your coaching information. Keras helps separate horizontal and vertical random shifting of coaching information by the width_shift_range and height_shift_range arguments.

Working this instance creates shifted variations of the digits. Once more, this isn’t required for MNIST because the handwritten digits are already centered, however you may see how this may be helpful on extra advanced drawback domains.

Random shifted MNIST photos

Random Flips

One other augmentation to your picture information that may enhance efficiency on massive and complicated issues is to create random flips of photos in your coaching information.

Keras helps random flipping alongside each the vertical and horizontal axes utilizing the vertical_flip and horizontal_flip arguments.

Working this instance, you may see flipped digits. Flipping digits just isn’t helpful as they’ll at all times have the proper left and proper orientation, however this can be helpful for issues with images of objects in a scene that may have a assorted orientation.

Randomly flipped MNIST photos

Saving Augmented Pictures to File

The information preparation and augmentation are carried out simply in time by Keras.

That is environment friendly when it comes to reminiscence, however you might require the precise photos used throughout coaching. For instance, maybe you wish to use them with a special software program package deal later or solely generate them as soon as and use them on a number of totally different deep studying fashions or configurations.

Keras lets you save the pictures generated throughout coaching. The listing, filename prefix, and picture file sort may be specified to the circulation() operate earlier than coaching. Then, throughout coaching, the generated photos might be written to the file.

The instance beneath demonstrates this and writes 9 photos to a “photos” subdirectory with the prefix “aug” and the file sort of PNG.

Working the instance, you may see that photos are solely written when they’re generated.

Augmented MNIST Images Saved To File

Augmented MNIST photos saved to file

Ideas for Augmenting Picture Knowledge with Keras

Picture information is exclusive in which you could overview the info and remodeled copies of the info and shortly get an thought of how the mannequin could understand it.

Beneath are some ideas for getting probably the most from picture information preparation and augmentation for deep studying.

  • Overview Dataset. Take a while to overview your dataset in nice element. Have a look at the pictures. Be aware of picture preparation and augmentations that may profit the coaching technique of your mannequin, corresponding to the necessity to deal with totally different shifts, rotations, or flips of objects within the scene.
  • Overview Augmentations. Overview pattern photos after the augmentation has been carried out. It’s one factor to intellectually know what picture transforms you might be utilizing; it’s a very totally different factor to take a look at examples. Overview photos each with particular person augmentations you might be utilizing in addition to the complete set of augmentations you propose to make use of. You might even see methods to simplify or additional improve your mannequin coaching course of.
  • Consider a Suite of Transforms. Attempt multiple picture information preparation and augmentation scheme. Typically you may be stunned by the outcomes of an information preparation scheme you didn’t assume could be useful.


On this submit, you found picture information preparation and augmentation.

You found a spread of strategies you should utilize simply in Python with Keras for deep studying fashions. You realized about:

  • The ImageDataGenerator API in Keras for producing remodeled photos simply in time
  • Pattern-wise and Characteristic-wise pixel standardization
  • The ZCA whitening rework
  • Random rotations, shifts, and flips of photos
  • save remodeled photos to file for later reuse

Do you might have any questions on picture information augmentation or this submit? Ask your questions within the feedback, and I’ll do my greatest to reply.

Develop Deep Studying Tasks with Python!

Deep Learning with Python

 What If You Might Develop A Community in Minutes

…with just some strains of Python

Uncover how in my new E-book:

Deep Studying With Python

It covers end-to-end tasks on subjects like:

Multilayer PerceptronsConvolutional Nets and Recurrent Neural Nets, and extra…

Lastly Deliver Deep Studying To

Your Personal Tasks

Skip the Teachers. Simply Outcomes.

See What’s Inside


Please enter your comment!
Please enter your name here