Update tensorflow patch to allow tf-text nightlies to build.

The patch was out-dated.  Applied the changes manually from the latest nightly and re-exported the patch.

FUTURE_COPYBARA_INTEGRATE_REVIEW=#1266 from 8bitmp3:fix-transformer fca8889
PiperOrigin-RevId: 634504731

WORKSPACE

@@ -58,10 +58,10 @@ http_archive(
     name = "org_tensorflow",
     patch_args = ["-p1"],
     patches = ["//third_party/tensorflow:tf.patch"],
-    strip_prefix = "tensorflow-d17c801006947b240ec4b8caf232c39b6a24718a",
-    sha256 = "1a32ed7b5ea090db114008ea382c1e1beda622ffd4c62582f2f906cb10ee6290",
+    strip_prefix = "tensorflow-f6b72954734f8304bfb83228bd8406a3ba3394f4",
+    sha256 = "15df197aace44fe2c67e6e22f930cf76f45d9e6ac1291e7c9ce8dd0dcc26e9a5",
     urls = [
-        "https://github.com/tensorflow/tensorflow/archive/d17c801006947b240ec4b8caf232c39b6a24718a.zip"
+        "https://github.com/tensorflow/tensorflow/archive/f6b72954734f8304bfb83228bd8406a3ba3394f4.zip"
     ],
 )
 

docs/tutorials/transformer.ipynb

@@ -93,7 +93,7 @@
       "source": [
         "This tutorial demonstrates how to create and train a [sequence-to-sequence](https://developers.google.com/machine-learning/glossary#sequence-to-sequence-task) [Transformer](https://developers.google.com/machine-learning/glossary#Transformer) model to translate [Portuguese into English](https://www.tensorflow.org/datasets/catalog/ted_hrlr_translate#ted_hrlr_translatept_to_en). The Transformer was originally proposed in [\"Attention is all you need\"](https://arxiv.org/abs/1706.03762) by Vaswani et al. (2017).\n",
         "\n",
-        "Transformers are deep neural networks that replace CNNs and RNNs with [self-attention](https://developers.google.com/machine-learning/glossary#self-attention). Self attention allows Transformers to easily transmit information across the input sequences.\n",
+        "Transformers are deep neural networks that replace CNNs and RNNs with [self-attention](https://developers.google.com/machine-learning/glossary#self-attention). Self-attention allows Transformers to easily transmit information across the input sequences.\n",
         "\n",
         "As explained in the [Google AI Blog post](https://ai.googleblog.com/2017/08/transformer-novel-neural-network.html):\n",
         "\n",
@@ -138,7 +138,7 @@
         "To get the most out of this tutorial, it helps if you know about [the basics of text generation](./text_generation.ipynb) and attention mechanisms. \n",
         "\n",
         "A Transformer is a sequence-to-sequence encoder-decoder model similar to the model in the [NMT with attention tutorial](https://www.tensorflow.org/text/tutorials/nmt_with_attention).\n",
-        "A single-layer Transformer takes a little more code to write, but is almost identical to that encoder-decoder RNN model. The only difference is that the RNN layers are replaced with self attention layers.\n",
+        "A single-layer Transformer takes a little more code to write, but is almost identical to that encoder-decoder RNN model. The only difference is that the RNN layers are replaced with self-attention layers.\n",
         "This tutorial builds a 4-layer Transformer which is larger and more powerful, but not fundamentally more complex."
       ]
     },
@@ -186,8 +186,8 @@
         "## Why Transformers are significant\n",
         "\n",
         "- Transformers excel at modeling sequential data, such as natural language.\n",
-        "- Unlike the [recurrent neural networks (RNNs)](./text_generation.ipynb), Transformers are parallelizable. This makes them efficient on hardware like GPUs and TPUs. The main reasons is that Transformers replaced recurrence with attention, and computations can happen simultaneously. Layer outputs can be computed in parallel, instead of a series like an RNN.\n",
-        "- Unlike [RNNs](https://www.tensorflow.org/guide/keras/rnn) (like [seq2seq, 2014](https://arxiv.org/abs/1409.3215)) or [convolutional neural networks (CNNs)](https://www.tensorflow.org/tutorials/images/cnn) (for example, [ByteNet](https://arxiv.org/abs/1610.10099)), Transformers are able to capture distant or long-range contexts and dependencies in the data between distant positions in the input or output sequences. Thus, longer connections can be learned. Attention allows each location to have access to the entire input at each layer, while in RNNs and CNNs, the information needs to pass through many processing steps to move a long distance, which makes it harder to learn.\n",
+        "- Unlike [recurrent neural networks (RNNs)](./text_generation.ipynb), Transformers are parallelizable. This makes them efficient on hardware like GPUs and TPUs. The main reasons is that Transformers replaced recurrence with attention, and computations can happen simultaneously. Layer outputs can be computed in parallel, instead of a series like an RNN.\n",
+        "- Unlike [RNNs](https://www.tensorflow.org/guide/keras/rnn) (such as [seq2seq, 2014](https://arxiv.org/abs/1409.3215)) or [convolutional neural networks (CNNs)](https://www.tensorflow.org/tutorials/images/cnn) (for example, [ByteNet](https://arxiv.org/abs/1610.10099)), Transformers are able to capture distant or long-range contexts and dependencies in the data between distant positions in the input or output sequences. Thus, longer connections can be learned. Attention allows each location to have access to the entire input at each layer, while in RNNs and CNNs, the information needs to pass through many processing steps to move a long distance, which makes it harder to learn.\n",
         "- Transformers make no assumptions about the temporal/spatial relationships across the data. This is ideal for processing a set of objects (for example, [StarCraft units](https://www.deepmind.com/blog/alphastar-mastering-the-real-time-strategy-game-starcraft-ii)).\n",
         "\n",
         "\u003cimg src=\"https://www.tensorflow.org/images/tutorials/transformer/encoder_self_attention_distribution.png\" width=\"800\" alt=\"Encoder self-attention distribution for the word it from the 5th to the 6th layer of a Transformer trained on English-to-French translation\"\u003e\n",
@@ -1007,8 +1007,8 @@
       },
       "outputs": [],
       "source": [
-        "embed_pt = PositionalEmbedding(vocab_size=tokenizers.pt.get_vocab_size(), d_model=512)\n",
-        "embed_en = PositionalEmbedding(vocab_size=tokenizers.en.get_vocab_size(), d_model=512)\n",
+        "embed_pt = PositionalEmbedding(vocab_size=tokenizers.pt.get_vocab_size().numpy(), d_model=512)\n",
+        "embed_en = PositionalEmbedding(vocab_size=tokenizers.en.get_vocab_size().numpy(), d_model=512)\n",
         "\n",
         "pt_emb = embed_pt(pt)\n",
         "en_emb = embed_en(en)"
@@ -1340,7 +1340,7 @@
         "id": "J6qrQxSpv34R"
       },
       "source": [
-        "### The global self attention layer"
+        "### The global self-attention layer"
       ]
     },
     {
@@ -1360,7 +1360,7 @@
       "source": [
         "\u003ctable\u003e\n",
         "\u003ctr\u003e\n",
-        "  \u003cth colspan=1\u003eThe global self attention layer\u003c/th\u003e\n",
+        "  \u003cth colspan=1\u003eThe global self-attention layer\u003c/th\u003e\n",
         "\u003ctr\u003e\n",
         "\u003ctr\u003e\n",
         "  \u003ctd\u003e\n",
@@ -1378,7 +1378,7 @@
       "source": [
         "Since the context sequence is fixed while the translation is being generated, information is allowed to flow in both directions. \n",
         "\n",
-        "Before Transformers and self attention, models commonly used RNNs or CNNs to do this task:"
+        "Before Transformers and self-attention, models commonly used RNNs or CNNs to do this task:"
       ]
     },
     {
@@ -1415,7 +1415,7 @@
         "- The RNN allows information to flow all the way across the sequence, but it passes through many processing steps to get there (limiting gradient flow). These RNN steps have to be run sequentially and so the RNN is less able to take advantage of modern parallel devices.\n",
         "- In the CNN each location can be processed in parallel, but it only provides a limited receptive field. The receptive field only grows linearly with the number of CNN layers,  You need to stack a number of Convolution layers to transmit information across the sequence ([Wavenet](https://arxiv.org/abs/1609.03499) reduces this problem by using dilated convolutions).\n",
         "\n",
-        "The global self attention layer on the other hand lets every sequence element directly access every other sequence element, with only a few operations, and all the outputs can be computed in parallel. \n",
+        "The global self-attention layer on the other hand lets every sequence element directly access every other sequence element, with only a few operations, and all the outputs can be computed in parallel. \n",
         "\n",
         "To implement this layer you just need to pass the target sequence, `x`, as both the `query`, and `value` arguments to the `mha` layer: "
       ]
@@ -1470,7 +1470,7 @@
       "source": [
         "\u003ctable\u003e\n",
         "\u003ctr\u003e\n",
-        "  \u003cth colspan=1\u003eThe global self attention layer\u003c/th\u003e\n",
+        "  \u003cth colspan=1\u003eThe global self-attention layer\u003c/th\u003e\n",
         "\u003ctr\u003e\n",
         "\u003ctr\u003e\n",
         "  \u003ctd\u003e\n",
@@ -1499,7 +1499,7 @@
       "source": [
         "\u003ctable\u003e\n",
         "\u003ctr\u003e\n",
-        "  \u003cth colspan=1\u003eThe global self attention layer\u003c/th\u003e\n",
+        "  \u003cth colspan=1\u003eThe global self-attention layer\u003c/th\u003e\n",
         "\u003ctr\u003e\n",
         "\u003ctr\u003e\n",
         "  \u003ctd\u003e\n",
@@ -1515,7 +1515,7 @@
         "id": "Yq4NtLymD99-"
       },
       "source": [
-        "### The causal self attention layer"
+        "### The causal self-attention layer"
       ]
     },
     {
@@ -1524,7 +1524,7 @@
         "id": "VufkgF7caLze"
       },
       "source": [
-        "This layer does a similar job as the global self attention layer, for the output sequence:"
+        "This layer does a similar job as the global self-attention layer, for the output sequence:"
       ]
     },
     {
@@ -1535,7 +1535,7 @@
       "source": [
         "\u003ctable\u003e\n",
         "\u003ctr\u003e\n",
-        "  \u003cth colspan=1\u003eThe causal self attention layer\u003c/th\u003e\n",
+        "  \u003cth colspan=1\u003eThe causal self-attention layer\u003c/th\u003e\n",
         "\u003ctr\u003e\n",
         "\u003ctr\u003e\n",
         "  \u003ctd\u003e\n",
@@ -1551,7 +1551,7 @@
         "id": "0AtF1HYFEOYf"
       },
       "source": [
-        "This needs to be handled differently from the encoder's global self attention layer.  \n",
+        "This needs to be handled differently from the encoder's global self-attention layer.  \n",
         "\n",
         "Like the [text generation tutorial](https://www.tensorflow.org/text/tutorials/text_generation), and the [NMT with attention](https://www.tensorflow.org/text/tutorials/nmt_with_attention) tutorial, Transformers are an \"autoregressive\" model: They generate the text one token at a time and feed that output back to the input. To make this _efficient_, these models ensure that the output for each sequence element only depends on the previous sequence elements; the models are \"causal\"."
       ]
@@ -1608,7 +1608,7 @@
         "id": "WLYfIa8eiYgk"
       },
       "source": [
-        "To build a causal self attention layer, you need to use an appropriate mask when computing the attention scores and summing the attention `value`s.\n",
+        "To build a causal self-attention layer, you need to use an appropriate mask when computing the attention scores and summing the attention `value`s.\n",
         "\n",
         "This is taken care of automatically if you pass `use_causal_mask = True` to the `MultiHeadAttention` layer when you call it:"
       ]
@@ -1650,7 +1650,7 @@
       "source": [
         "\u003ctable\u003e\n",
         "\u003ctr\u003e\n",
-        "  \u003cth colspan=1\u003eThe causal self attention layer\u003c/th\u003e\n",
+        "  \u003cth colspan=1\u003eThe causal self-attention layer\u003c/th\u003e\n",
         "\u003ctr\u003e\n",
         "\u003ctr\u003e\n",
         "  \u003ctd\u003e\n",
@@ -1679,7 +1679,7 @@
       "source": [
         "\u003ctable\u003e\n",
         "\u003c/tr\u003e\n",
-        "  \u003cth colspan=1\u003eThe causal self attention layer\u003c/th\u003e\n",
+        "  \u003cth colspan=1\u003eThe causal self-attention layer\u003c/th\u003e\n",
         "\u003ctr\u003e\n",
         "\u003ctr\u003e\n",
         "  \u003ctd\u003e\n",

tensorflow_text/BUILD

@@ -1396,9 +1396,7 @@ py_tf_text_library(
     cc_op_kernels = [
         "//tensorflow_text/core/kernels:utf8_binarize_kernel",
     ],
-    visibility = [
-        "//visibility:private",  # Only private by automation, not intent. Owner may accept CLs adding visibility. See go/scheuklappen#explicit-private.
-    ],
+    visibility = ["//visibility:private"],
     deps = [
         # python/framework:ops tensorflow dep,
         # python/ops:array_ops tensorflow dep,

tensorflow_text/core/kernels/BUILD

@@ -1,7 +1,8 @@
-# Kernels for tf.text ops.
-# [internal] load cc_proto_library.bzl
+"""Kernels for tf.text ops."""
+
 load("@flatbuffers//:build_defs.bzl", "flatbuffer_cc_library")
 load("//tensorflow_text:tftext.bzl", "tf_cc_library", "tflite_cc_library")
+# [internal] load cc_proto_library.bzl
 
 licenses(["notice"])
 

third_party/tensorflow/tf.patch

@@ -1,33 +1,34 @@
 diff --git a/tensorflow/tools/toolchains/cpus/aarch64/aarch64_compiler_configure.bzl b/tensorflow/tools/toolchains/cpus/aarch64/aarch64_compiler_configure.bzl
-index a2bdd6a7eed..ec25c23d8d4 100644
+index 00cd6983ca3..d9c5ef16f9b 100644
 --- a/tensorflow/tools/toolchains/cpus/aarch64/aarch64_compiler_configure.bzl
 +++ b/tensorflow/tools/toolchains/cpus/aarch64/aarch64_compiler_configure.bzl
-@@ -2,7 +2,7 @@
+@@ -1,7 +1,7 @@
+ """Configurations of AARCH64 builds used with Docker container."""
 
  load("//tensorflow/tools/toolchains:cpus/aarch64/aarch64.bzl", "remote_aarch64_configure")
- load("//third_party/remote_config:remote_platform_configure.bzl", "remote_platform_configure")
 -load("//third_party/py:python_configure.bzl", "remote_python_configure")
 +load("//third_party/py/non_hermetic:python_configure.bzl", "remote_python_configure")
+ load("//third_party/remote_config:remote_platform_configure.bzl", "remote_platform_configure")
 
  def ml2014_tf_aarch64_configs(name_container_map, env):
-     for name, container in name_container_map.items():
 diff --git a/tensorflow/tools/toolchains/remote_config/rbe_config.bzl b/tensorflow/tools/toolchains/remote_config/rbe_config.bzl
-index 9f71a414bf7..57f70752323 100644
+index ae776c2a2fd..108e79edbd7 100644
 --- a/tensorflow/tools/toolchains/remote_config/rbe_config.bzl
 +++ b/tensorflow/tools/toolchains/remote_config/rbe_config.bzl
-@@ -1,6 +1,6 @@
- """Macro that creates external repositories for remote config."""
-
--load("//third_party/py:python_configure.bzl", "local_python_configure", "remote_python_configure")
-+load("//third_party/py/non_hermetic:python_configure.bzl", "local_python_configure", "remote_python_configure")
+@@ -4,7 +4,7 @@ load("//tensorflow/tools/toolchains/remote_config:containers.bzl", "containers")
  load("//third_party/gpus:cuda_configure.bzl", "remote_cuda_configure")
- load("//third_party/nccl:nccl_configure.bzl", "remote_nccl_configure")
  load("//third_party/gpus:rocm_configure.bzl", "remote_rocm_configure")
+ load("//third_party/nccl:nccl_configure.bzl", "remote_nccl_configure")
+-load("//third_party/py:python_configure.bzl", "local_python_configure", "remote_python_configure")
++load("//third_party/py/non_hermetic:python_configure.bzl", "local_python_configure", "remote_python_configure")
+ load("//third_party/remote_config:remote_platform_configure.bzl", "remote_platform_configure")
+ load("//third_party/tensorrt:tensorrt_configure.bzl", "remote_tensorrt_configure")
+
 diff --git a/tensorflow/workspace2.bzl b/tensorflow/workspace2.bzl
-index 056df85ffdb..7422baf8c59 100644
+index 77eea2ac869..54a3ec2fed6 100644
 --- a/tensorflow/workspace2.bzl
 +++ b/tensorflow/workspace2.bzl
-@@ -37,7 +37,7 @@ load("//third_party/nasm:workspace.bzl", nasm = "repo")
+@@ -44,7 +44,7 @@ load("//third_party/nasm:workspace.bzl", nasm = "repo")
  load("//third_party/nccl:nccl_configure.bzl", "nccl_configure")
  load("//third_party/opencl_headers:workspace.bzl", opencl_headers = "repo")
  load("//third_party/pasta:workspace.bzl", pasta = "repo")
@@ -37,15 +38,15 @@ index 056df85ffdb..7422baf8c59 100644
  load("//third_party/pybind11_abseil:workspace.bzl", pybind11_abseil = "repo")
  load("//third_party/pybind11_bazel:workspace.bzl", pybind11_bazel = "repo")
 diff --git a/third_party/py/non_hermetic/python_configure.bzl b/third_party/py/non_hermetic/python_configure.bzl
-index 300cbfb6c71..09d98505dd9 100644
+index 89732c3e33d..4ac1c8f5c04 100644
 --- a/third_party/py/non_hermetic/python_configure.bzl
 +++ b/third_party/py/non_hermetic/python_configure.bzl
-@@ -206,7 +206,7 @@ def _create_local_python_repository(repository_ctx):
+@@ -203,7 +203,7 @@ def _create_local_python_repository(repository_ctx):
      # Resolve all labels before doing any real work. Resolving causes the
      # function to be restarted with all previous state being lost. This
      # can easily lead to a O(n^2) runtime in the number of labels.
 -    build_tpl = repository_ctx.path(Label("//third_party/py:BUILD.tpl"))
 +    build_tpl = repository_ctx.path(Label("//third_party/py/non_hermetic:BUILD.tpl"))
-
+ 
      python_bin = get_python_bin(repository_ctx)
-     _check_python_bin(repository_ctx, python_bin)
+     _check_python_bin(repository_ctx, python_bin)