Deep Learning and AI

Update Alert: TensorFlow 2.7

November 8, 2021 • 9 min read

A Look at TensorFlow 2.7

Google released TensorFlow 2.7 last week which is a usability and performance improvement focused release.

Highlights include improvements to the debugging experience, support for auto-sharding in the data service, Keras enhancements, an experimental API to support conversion from Jax models to TensorFlow Lite, and more.

You can view the full list of changes on the TensorFlow GitHub page (and download and install the latest version): TensorFlow 2.7.0

Let's take a closer look at some of these features.

Improved Debugging Experience

Previously, TensorFlow error stack traces involved many internal frames, which could be challenging to read through, while not being actionable for end users. With version 2.7, TensorFlow filters internal frames in most errors that it raises, to keep stack traces short, readable, and focused on what's actionable for end users (their own code).

This behavior can be disabled by calling tf.debugging.disable_traceback_filtering(), and can be re-enabled via tf.debugging.enable_traceback_filtering(). If you are debugging a TensorFlow-internal issue (e.g. to prepare a TensorFlow PR), make sure to disable traceback filtering. You can check whether this feature is currently enabled by calling tf.debugging.is_traceback_filtering_enabled().

Note that this feature is only available with Python 3.7 or higher.

TensorFlow Data

TensorFlow data service now supports auto-sharding. Users specify the sharding policy with tf.data.experimental.service.ShardingPolicy enum. It can be one of OFF (equivalent to today's "parallel_epochs" mode), DYNAMIC (equivalent to today's "distributed_epoch" mode), or one of the static sharding policies: FILE, DATA, FILE_OR_DATA, or HINT (corresponding to values of tf.data.experimental.AutoShardPolicy).

Static sharding (auto-sharding) requires the number of tf.data service workers be fixed. Users need to specify the worker addresses in tensorflow.data.experimental.DispatcherConfig.

Keras Improvements

Keras layers now includes a public convolution_op method. This method can be used to simplify the implementation of Conv subclasses. There are two primary ways to use this new method. The first is to use the method directly in your own call method:

  class StandardizedConv2D(tf.keras.layers.Conv2D):
    def call(self, inputs):
      mean, var = tf.nn.moments(self.kernel, axes=[0, 1, 2], keepdims=True)
      return self.convolution_op(inputs, (self.kernel - mean) / tf.sqrt(var + 1e-10))

Alternatively, you can override convolution_op:

  class StandardizedConv2D(tf.keras.Layer):
    def convolution_op(self, inputs, kernel):
      mean, var = tf.nn.moments(kernel, axes=[0, 1, 2], keepdims=True)
      # Author code uses std + 1e-5
      return super().convolution_op(inputs, (kernel - mean) / tf.sqrt(var + 1e-10))

Other Keras enhancements include adding merge_state() method to tf.keras.metrics.Metric for use in distributed computations; and, adding sparse and ragged options to tf.keras.layers.TextVectorization to allow for SparseTensor and RaggedTensor outputs from the layer.

New Style Variables

Introduce the tf.compat.v1.keras.utils.track_tf1_style_variables decorator, which enables using large classes of tf1-style variable_scope, get_variable, and compat.v1.layer-based components from within TF2 models running with TF2 behavior enabled.

Security Fixes

There were also a large number of security fixes included. TensorFlow 2.7:

Fixes a code injection issue in saved_model_cli (CVE-2021-41228)
Fixes a vulnerability due to use of uninitialized value in Tensorflow (CVE-2021-41225)
Fixes a heap OOB in FusedBatchNorm kernels (CVE-2021-41223)
Fixes an arbitrary memory read in ImmutableConst (CVE-2021-41227)
Fixes a heap OOB in SparseBinCount (CVE-2021-41226)
Fixes a heap OOB in SparseFillEmptyRows (CVE-2021-41224)
Fixes a segfault due to negative splits in SplitV (CVE-2021-41222)
Fixes segfaults and vulnerabilities caused by accesses to invalid memory during shape inference in Cudnn* ops (CVE-2021-41221)
Fixes a null pointer exception when Exit node is not preceded by Enter op (CVE-2021-41217)
Fixes an integer division by 0 in tf.raw_ops.AllToAll (CVE-2021-41218)
Fixes a use after free and a memory leak in CollectiveReduceV2 (CVE-2021-41220)
Fixes an undefined behavior via nullptr reference binding in sparse matrix multiplication (CVE-2021-41219)
Fixes a heap buffer overflow in Transpose (CVE-2021-41216)
Prevents deadlocks arising from mutually recursive tf.function objects (CVE-2021-41213)
Fixes a null pointer exception in DeserializeSparse (CVE-2021-41215)
Fixes an undefined behavior arising from reference binding to nullptr in tf.ragged.cross (CVE-2021-41214)
Fixes a heap OOB read in tf.ragged.cross (CVE-2021-41212)
Fixes a heap OOB in shape inference for QuantizeV2 (CVE-2021-41211)
Fixes a heap OOB read in all tf.raw_ops.QuantizeAndDequantizeV* ops (CVE-2021-41205)
Fixes an FPE in ParallelConcat (CVE-2021-41207)
Fixes FPE issues in convolutions with zero size filters (CVE-2021-41209)
Fixes a heap OOB read in tf.raw_ops.SparseCountSparseOutput (CVE-2021-41210)
Fixes vulnerabilities caused by incomplete validation in boosted trees code (CVE-2021-41208)
Fixes vulnerabilities caused by incomplete validation of shapes in multiple TF ops (CVE-2021-41206)
Fixes a segfault produced while copying constant resource tensor (CVE-2021-41204)
Fixes a vulnerability caused by unitialized access in EinsumHelper::ParseEquation (CVE-2021-41201)
Fixes several vulnerabilities and segfaults caused by missing validation during checkpoint loading (CVE-2021-41203)
Fixes an overflow producing a crash in tf.range (CVE-2021-41202)
Fixes an overflow producing a crash in tf.image.resize when size is large (CVE-2021-41199)
Fixes an overflow producing a crash in tf.tile when tiling tensor is large (CVE-2021-41198)
Fixes a vulnerability produced due to incomplete validation in tf.summary.create_file_writer (CVE-2021-41200)
Fixes multiple crashes due to overflow and CHECK-fail in ops with large tensor shapes (CVE-2021-41197)
Fixes a crash in max_pool3d when size argument is 0 or negative (CVE-2021-41196)
Fixes a crash in tf.math.segment_* operations (CVE-2021-41195)
Updates curl to 7.78.0 to handle CVE-2021-22922, CVE-2021-22923, CVE-2021-22924, CVE-2021-22925, and CVE-2021-22926.

Feel free to contact us if you have any questions or take a look at our Deep Learning Solutions if you're interested in a workstation or server to run TensorFlow on.

Blog